on 07c8ca8 help sign in
searx/docs/dev/engine_overview.rst
267 lines 10 kB history raw
2.. _engines-dev:
4===============
5Engine overview
6===============
8.. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine
10searx is a metasearch-engine_, so it uses different search engines to provide
11better results.
13Because there is no general search API which could be used for every search
14engine, an adapter has to be built between searx and the external search
15engines. Adapters are stored under the folder :origin:`searx/engines`.
17.. contents::
18 :depth: 3
19 :backlinks: entry
21general engine configuration
22============================
24It is required to tell searx the type of results the engine provides. The
25arguments can be set in the engine file or in the settings file
26(normally ``settings.yml``). The arguments in the settings file override
27the ones in the engine file.
29It does not matter if an option is stored in the engine file or in the
30settings. However, the standard way is the following:
32.. _engine file:
34engine file
35-----------
37======================= =========== ===========================================
38argument type information
39======================= =========== ===========================================
40categories list pages, in which the engine is working
41paging boolean support multible pages
42language_support boolean support language choosing
43time_range_support boolean support search time range
44offline boolean engine runs offline
45======================= =========== ===========================================
47.. _engine settings:
49settings.yml
50------------
52======================= =========== ===========================================
53argument type information
54======================= =========== ===========================================
55name string name of search-engine
56engine string name of searx-engine
57 (filename without ``.py``)
58shortcut string shortcut of search-engine
59timeout string specific timeout for search-engine
60======================= =========== ===========================================
63overrides
64---------
66A few of the options have default values in the engine, but are often
67overwritten by the settings. If ``None`` is assigned to an option in the engine
68file, it has to be redefined in the settings, otherwise searx will not start
69with that engine.
71The naming of overrides is arbitrary. But the recommended overrides are the
72following:
74======================= =========== ===========================================
75argument type information
76======================= =========== ===========================================
77base_url string base-url, can be overwritten to use same
78 engine on other URL
79number_of_results int maximum number of results per request
80language string ISO code of language and country like en_US
81api_key string api-key if required by engine
82======================= =========== ===========================================
84example code
85------------
87.. code:: python
89 # engine dependent config
90 categories = ['general']
91 paging = True
92 language_support = True
95making a request
96================
98To perform a search an URL have to be specified. In addition to specifying an
99URL, arguments can be passed to the query.
101passed arguments
102----------------
104These arguments can be used to construct the search query. Furthermore,
105parameters with default value can be redefined for special purposes.
107====================== ============ ========================================================================
108argument type default-value, information
109====================== ============ ========================================================================
110url string ``''``
111method string ``'GET'``
112headers set ``{}``
113data set ``{}``
114cookies set ``{}``
115verify boolean ``True``
116headers.User-Agent string a random User-Agent
117category string current category, like ``'general'``
118started datetime current date-time
119pageno int current pagenumber
120language string specific language code like ``'en_US'``, or ``'all'`` if unspecified
121====================== ============ ========================================================================
123parsed arguments
124----------------
126The function ``def request(query, params):`` always returns the ``params``
127variable. Inside searx, the following paramters can be used to specify a search
128request:
130============ =========== =========================================================
131argument type information
132============ =========== =========================================================
133url string requested url
134method string HTTP request method
135headers set HTTP header information
136data set HTTP data information (parsed if ``method != 'GET'``)
137cookies set HTTP cookies
138verify boolean Performing SSL-Validity check
139============ =========== =========================================================
142example code
143------------
145.. code:: python
147 # search-url
148 base_url = 'https://example.com/'
149 search_string = 'search?{query}&page={page}'
151 # do search-request
152 def request(query, params):
153 search_path = search_string.format(
154 query=urlencode({'q': query}),
155 page=params['pageno'])
157 params['url'] = base_url + search_path
159 return params
162returned results
163================
165Searx is able to return results of different media-types. Currently the
166following media-types are supported:
168- default_
169- images_
170- videos_
171- torrent_
172- map_
174To set another media-type as default, the parameter ``template`` must be set to
175the desired type.
177default
178-------
180========================= =====================================================
181result-parameter information
182========================= =====================================================
183url string, url of the result
184title string, title of the result
185content string, general result-text
186publishedDate :py:class:`datetime.datetime`, time of publish
187========================= =====================================================
189images
190------
192To use this template, the parameter:
194========================= =====================================================
195result-parameter information
196========================= =====================================================
197template is set to ``images.html``
198url string, url to the result site
199title string, title of the result *(partly implemented)*
200content *(partly implemented)*
201publishedDate :py:class:`datetime.datetime`,
202 time of publish *(partly implemented)*
203img\_src string, url to the result image
204thumbnail\_src string, url to a small-preview image
205========================= =====================================================
207videos
208------
210========================= =====================================================
211result-parameter information
212========================= =====================================================
213template is set to ``videos.html``
214url string, url of the result
215title string, title of the result
216content *(not implemented yet)*
217publishedDate :py:class:`datetime.datetime`, time of publish
218thumbnail string, url to a small-preview image
219========================= =====================================================
221torrent
222-------
224.. _magnetlink: https://en.wikipedia.org/wiki/Magnet_URI_scheme
226========================= =====================================================
227result-parameter information
228========================= =====================================================
229template is set to ``torrent.html``
230url string, url of the result
231title string, title of the result
232content string, general result-text
233publishedDate :py:class:`datetime.datetime`,
234 time of publish *(not implemented yet)*
235seed int, number of seeder
236leech int, number of leecher
237filesize int, size of file in bytes
238files int, number of files
239magnetlink string, magnetlink_ of the result
240torrentfile string, torrentfile of the result
241========================= =====================================================
244map
245---
247========================= =====================================================
248result-parameter information
249========================= =====================================================
250url string, url of the result
251title string, title of the result
252content string, general result-text
253publishedDate :py:class:`datetime.datetime`, time of publish
254latitude latitude of result (in decimal format)
255longitude longitude of result (in decimal format)
256boundingbox boundingbox of result (array of 4. values
257 ``[lat-min, lat-max, lon-min, lon-max]``)
258geojson geojson of result (http://geojson.org)
259osm.type type of osm-object (if OSM-Result)
260osm.id id of osm-object (if OSM-Result)
261address.name name of object
262address.road street name of object
263address.house_number house number of object
264address.locality city, place of object
265address.postcode postcode of object
266address.country country of object
267========================= =====================================================