git.iko.soy/searx
on 6c6985a help sign in
searx/docs/dev/engine_overview.rst
271 lines 11 kB history raw
2.. _engines-dev:
4===============
5Engine overview
6===============
8.. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine
10searx is a metasearch-engine_, so it uses different search engines to provide
11better results.
13Because there is no general search API which could be used for every search
14engine, an adapter has to be built between searx and the external search
15engines. Adapters are stored under the folder :origin:`searx/engines`.
17.. contents::
18 :depth: 3
19 :backlinks: entry
21general engine configuration
22============================
24It is required to tell searx the type of results the engine provides. The
25arguments can be set in the engine file or in the settings file
26(normally ``settings.yml``). The arguments in the settings file override
27the ones in the engine file.
29It does not matter if an option is stored in the engine file or in the
30settings. However, the standard way is the following:
32.. _engine file:
34engine file
35-----------
37======================= =========== ===========================================
38argument type information
39======================= =========== ===========================================
40categories list pages, in which the engine is working
41paging boolean support multible pages
42language_support boolean support language choosing
43time_range_support boolean support search time range
44offline boolean engine runs offline
45======================= =========== ===========================================
47.. _engine settings:
49settings.yml
50------------
52======================= =========== =============================================
53argument type information
54======================= =========== =============================================
55name string name of search-engine
56engine string name of searx-engine
57 (filename without ``.py``)
58shortcut string shortcut of search-engine
59timeout string specific timeout for search-engine
60display_error_messages boolean display error messages on the web UI
61proxies dict set proxies for a specific engine
62 (e.g. ``proxies : {http: socks5://proxy:port,
63 https: socks5://proxy:port}``)
64======================= =========== =============================================
67overrides
68---------
70A few of the options have default values in the engine, but are often
71overwritten by the settings. If ``None`` is assigned to an option in the engine
72file, it has to be redefined in the settings, otherwise searx will not start
73with that engine.
75The naming of overrides is arbitrary. But the recommended overrides are the
76following:
78======================= =========== ===========================================
79argument type information
80======================= =========== ===========================================
81base_url string base-url, can be overwritten to use same
82 engine on other URL
83number_of_results int maximum number of results per request
84language string ISO code of language and country like en_US
85api_key string api-key if required by engine
86======================= =========== ===========================================
88example code
89------------
91.. code:: python
93 # engine dependent config
94 categories = ['general']
95 paging = True
96 language_support = True
99making a request
100================
102To perform a search an URL have to be specified. In addition to specifying an
103URL, arguments can be passed to the query.
105passed arguments
106----------------
108These arguments can be used to construct the search query. Furthermore,
109parameters with default value can be redefined for special purposes.
111====================== ============ ========================================================================
112argument type default-value, information
113====================== ============ ========================================================================
114url string ``''``
115method string ``'GET'``
116headers set ``{}``
117data set ``{}``
118cookies set ``{}``
119verify boolean ``True``
120headers.User-Agent string a random User-Agent
121category string current category, like ``'general'``
122started datetime current date-time
123pageno int current pagenumber
124language string specific language code like ``'en_US'``, or ``'all'`` if unspecified
125====================== ============ ========================================================================
127parsed arguments
128----------------
130The function ``def request(query, params):`` always returns the ``params``
131variable. Inside searx, the following paramters can be used to specify a search
132request:
134============ =========== =========================================================
135argument type information
136============ =========== =========================================================
137url string requested url
138method string HTTP request method
139headers set HTTP header information
140data set HTTP data information (parsed if ``method != 'GET'``)
141cookies set HTTP cookies
142verify boolean Performing SSL-Validity check
143============ =========== =========================================================
146example code
147------------
149.. code:: python
151 # search-url
152 base_url = 'https://example.com/'
153 search_string = 'search?{query}&page={page}'
155 # do search-request
156 def request(query, params):
157 search_path = search_string.format(
158 query=urlencode({'q': query}),
159 page=params['pageno'])
161 params['url'] = base_url + search_path
163 return params
166returned results
167================
169Searx is able to return results of different media-types. Currently the
170following media-types are supported:
172- default_
173- images_
174- videos_
175- torrent_
176- map_
178To set another media-type as default, the parameter ``template`` must be set to
179the desired type.
181default
182-------
184========================= =====================================================
185result-parameter information
186========================= =====================================================
187url string, url of the result
188title string, title of the result
189content string, general result-text
190publishedDate :py:class:`datetime.datetime`, time of publish
191========================= =====================================================
193images
194------
196To use this template, the parameter:
198========================= =====================================================
199result-parameter information
200========================= =====================================================
201template is set to ``images.html``
202url string, url to the result site
203title string, title of the result *(partly implemented)*
204content *(partly implemented)*
205publishedDate :py:class:`datetime.datetime`,
206 time of publish *(partly implemented)*
207img\_src string, url to the result image
208thumbnail\_src string, url to a small-preview image
209========================= =====================================================
211videos
212------
214========================= =====================================================
215result-parameter information
216========================= =====================================================
217template is set to ``videos.html``
218url string, url of the result
219title string, title of the result
220content *(not implemented yet)*
221publishedDate :py:class:`datetime.datetime`, time of publish
222thumbnail string, url to a small-preview image
223========================= =====================================================
225torrent
226-------
228.. _magnetlink: https://en.wikipedia.org/wiki/Magnet_URI_scheme
230========================= =====================================================
231result-parameter information
232========================= =====================================================
233template is set to ``torrent.html``
234url string, url of the result
235title string, title of the result
236content string, general result-text
237publishedDate :py:class:`datetime.datetime`,
238 time of publish *(not implemented yet)*
239seed int, number of seeder
240leech int, number of leecher
241filesize int, size of file in bytes
242files int, number of files
243magnetlink string, magnetlink_ of the result
244torrentfile string, torrentfile of the result
245========================= =====================================================
248map
249---
251========================= =====================================================
252result-parameter information
253========================= =====================================================
254url string, url of the result
255title string, title of the result
256content string, general result-text
257publishedDate :py:class:`datetime.datetime`, time of publish
258latitude latitude of result (in decimal format)
259longitude longitude of result (in decimal format)
260boundingbox boundingbox of result (array of 4. values
261 ``[lat-min, lat-max, lon-min, lon-max]``)
262geojson geojson of result (http://geojson.org)
263osm.type type of osm-object (if OSM-Result)
264osm.id id of osm-object (if OSM-Result)
265address.name name of object
266address.road street name of object
267address.house_number house number of object
268address.locality city, place of object
269address.postcode postcode of object
270address.country country of object
271========================= =====================================================