on f9faafa help sign in
searx/docs/dev/engine_overview.rst
268 lines 10 kB history raw
2.. _engines-dev:
4===============
5Engine overview
6===============
8.. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine
10searx is a metasearch-engine_, so it uses different search engines to provide
11better results.
13Because there is no general search API which could be used for every search
14engine, an adapter has to be built between searx and the external search
15engines. Adapters are stored under the folder :origin:`searx/engines`.
17.. contents::
18 :depth: 3
19 :backlinks: entry
21general engine configuration
22============================
24It is required to tell searx the type of results the engine provides. The
25arguments can be set in the engine file or in the settings file
26(normally ``settings.yml``). The arguments in the settings file override
27the ones in the engine file.
29It does not matter if an option is stored in the engine file or in the
30settings. However, the standard way is the following:
32.. _engine file:
34engine file
35-----------
37======================= =========== ===========================================
38argument type information
39======================= =========== ===========================================
40categories list pages, in which the engine is working
41paging boolean support multible pages
42language_support boolean support language choosing
43time_range_support boolean support search time range
44offline boolean engine runs offline
45======================= =========== ===========================================
47.. _engine settings:
49settings.yml
50------------
52======================= =========== ===========================================
53argument type information
54======================= =========== ===========================================
55name string name of search-engine
56engine string name of searx-engine
57 (filename without ``.py``)
58shortcut string shortcut of search-engine
59timeout string specific timeout for search-engine
60display_error_messages boolean display error messages on the web UI
61======================= =========== ===========================================
64overrides
65---------
67A few of the options have default values in the engine, but are often
68overwritten by the settings. If ``None`` is assigned to an option in the engine
69file, it has to be redefined in the settings, otherwise searx will not start
70with that engine.
72The naming of overrides is arbitrary. But the recommended overrides are the
73following:
75======================= =========== ===========================================
76argument type information
77======================= =========== ===========================================
78base_url string base-url, can be overwritten to use same
79 engine on other URL
80number_of_results int maximum number of results per request
81language string ISO code of language and country like en_US
82api_key string api-key if required by engine
83======================= =========== ===========================================
85example code
86------------
88.. code:: python
90 # engine dependent config
91 categories = ['general']
92 paging = True
93 language_support = True
96making a request
97================
99To perform a search an URL have to be specified. In addition to specifying an
100URL, arguments can be passed to the query.
102passed arguments
103----------------
105These arguments can be used to construct the search query. Furthermore,
106parameters with default value can be redefined for special purposes.
108====================== ============ ========================================================================
109argument type default-value, information
110====================== ============ ========================================================================
111url string ``''``
112method string ``'GET'``
113headers set ``{}``
114data set ``{}``
115cookies set ``{}``
116verify boolean ``True``
117headers.User-Agent string a random User-Agent
118category string current category, like ``'general'``
119started datetime current date-time
120pageno int current pagenumber
121language string specific language code like ``'en_US'``, or ``'all'`` if unspecified
122====================== ============ ========================================================================
124parsed arguments
125----------------
127The function ``def request(query, params):`` always returns the ``params``
128variable. Inside searx, the following paramters can be used to specify a search
129request:
131============ =========== =========================================================
132argument type information
133============ =========== =========================================================
134url string requested url
135method string HTTP request method
136headers set HTTP header information
137data set HTTP data information (parsed if ``method != 'GET'``)
138cookies set HTTP cookies
139verify boolean Performing SSL-Validity check
140============ =========== =========================================================
143example code
144------------
146.. code:: python
148 # search-url
149 base_url = 'https://example.com/'
150 search_string = 'search?{query}&page={page}'
152 # do search-request
153 def request(query, params):
154 search_path = search_string.format(
155 query=urlencode({'q': query}),
156 page=params['pageno'])
158 params['url'] = base_url + search_path
160 return params
163returned results
164================
166Searx is able to return results of different media-types. Currently the
167following media-types are supported:
169- default_
170- images_
171- videos_
172- torrent_
173- map_
175To set another media-type as default, the parameter ``template`` must be set to
176the desired type.
178default
179-------
181========================= =====================================================
182result-parameter information
183========================= =====================================================
184url string, url of the result
185title string, title of the result
186content string, general result-text
187publishedDate :py:class:`datetime.datetime`, time of publish
188========================= =====================================================
190images
191------
193To use this template, the parameter:
195========================= =====================================================
196result-parameter information
197========================= =====================================================
198template is set to ``images.html``
199url string, url to the result site
200title string, title of the result *(partly implemented)*
201content *(partly implemented)*
202publishedDate :py:class:`datetime.datetime`,
203 time of publish *(partly implemented)*
204img\_src string, url to the result image
205thumbnail\_src string, url to a small-preview image
206========================= =====================================================
208videos
209------
211========================= =====================================================
212result-parameter information
213========================= =====================================================
214template is set to ``videos.html``
215url string, url of the result
216title string, title of the result
217content *(not implemented yet)*
218publishedDate :py:class:`datetime.datetime`, time of publish
219thumbnail string, url to a small-preview image
220========================= =====================================================
222torrent
223-------
225.. _magnetlink: https://en.wikipedia.org/wiki/Magnet_URI_scheme
227========================= =====================================================
228result-parameter information
229========================= =====================================================
230template is set to ``torrent.html``
231url string, url of the result
232title string, title of the result
233content string, general result-text
234publishedDate :py:class:`datetime.datetime`,
235 time of publish *(not implemented yet)*
236seed int, number of seeder
237leech int, number of leecher
238filesize int, size of file in bytes
239files int, number of files
240magnetlink string, magnetlink_ of the result
241torrentfile string, torrentfile of the result
242========================= =====================================================
245map
246---
248========================= =====================================================
249result-parameter information
250========================= =====================================================
251url string, url of the result
252title string, title of the result
253content string, general result-text
254publishedDate :py:class:`datetime.datetime`, time of publish
255latitude latitude of result (in decimal format)
256longitude longitude of result (in decimal format)
257boundingbox boundingbox of result (array of 4. values
258 ``[lat-min, lat-max, lon-min, lon-max]``)
259geojson geojson of result (http://geojson.org)
260osm.type type of osm-object (if OSM-Result)
261osm.id id of osm-object (if OSM-Result)
262address.name name of object
263address.road street name of object
264address.house_number house number of object
265address.locality city, place of object
266address.postcode postcode of object
267address.country country of object
268========================= =====================================================