on master help sign in
add post about introducing offline engines
commit021e1da4c9a8eca7293ad62608cc55da5dfde73f authorNoémi Ványi <sitbackandwait@gmail.com> dateWed 23 Oct 2019, 13:06 (7 years ago) parentc1ccb25
4 files changed+69−1
docs/blog/blog.rst+1 docs/blog/intro-offline.rst+65 docs/conf.py+1−1 docs/dev/engine_overview.rst+2
docs/blog/blog.rst+1
@@ -6,3 +6,4 @@ Blog
66
77 python3
88 admin
9+ intro-offline
docs/blog/intro-offline.rstnew file+65
@@ -0,0 +1,65 @@
1+Preparation for offline engines
2+===============================
3+
4+Offline engines
5+---------------
6+
7+To extend the functionality of searx, offline engines are going to be introduced. An offline engine is an engine which does not need Internet connection to perform a search and does not use HTTP to communicate.
8+
9+Offline engines can be configured as online engines, by adding those to the `engines` list of `settings.yml`. Thus, searx finds the engine file and imports it.
10+
11+Example skeleton for the new engines:
12+
13+.. code:: python
14+
15+ from subprocess import PIPE, Popen
16+
17+ categories = ['general']
18+ offline = True
19+
20+ def init(settings):
21+ pass
22+
23+ def search(query, params):
24+ process = Popen(['ls', query], stdout=PIPE)
25+ return_code = process.wait()
26+ if return_code != 0:
27+ raise RuntimeError('non-zero return code', return_code)
28+
29+ results = []
30+ line = process.stdout.readline()
31+ while line:
32+ result = parse_line(line)
33+ results.append(results)
34+
35+ line = process.stdout.readline()
36+
37+ return results
38+
39+
40+Development progress
41+--------------------
42+
43+First, a proposal has been created as a Github issue. Then it was moved to the wiki as a design document. You can read it here: https://github.com/asciimoo/searx/wiki/Offline-engines
44+
45+In this development step, searx core was prepared to accept and perform offline searches. Offline search requests are scheduled together with regular offline requests.
46+
47+As offline searches can return arbitrary results depending on the engine, the current result templates were insufficient to present such results. Thus, a new template is introduced which is caplable of presenting arbitrary key value pairs as a table. You can check out the pull request for more details: https://github.com/asciimoo/searx/pull/1700
48+
49+Next steps
50+----------
51+
52+Today, it is possible to create/run an offline engine. However, it is going to be publicly available for everyone who knows the searx instance. So the next step is to introduce token based access for engines. This way administrators are able to limit the access to private engines.
53+
54+Acknowledgement
55+---------------
56+
57+This development was sponsored by `Search and Discovery Fund`_ of `NLnet Foundation`_ .
58+
59+.. _Search and Discovery Fund: https://nlnet.nl/discovery
60+.. _NLnet Foundation: https://nlnet.nl/
61+
62+
63+| Happy hacking.
64+| kvch // 2019.10.21 17:03
65+
docs/conf.py+1−1
@@ -49,7 +49,7 @@ master_doc = 'index'
4949
5050# General information about the project.
5151project = u'searx'
52−copyright = u'2015-2017, Adam Tauber'
52+copyright = u'2015-2019, Adam Tauber, Noémi Ványi'
5353author = u'Adam Tauber'
5454
5555# The version info for the project you're documenting, acts as replacement for
docs/dev/engine_overview.rst+2
@@ -41,6 +41,8 @@ engine file
4141+----------------------+-----------+-----------------------------------------+
4242| time\_range\_support | boolean | support search time range |
4343+----------------------+-----------+-----------------------------------------+
44+| offline | boolean | engine runs offline |
45++----------------------+-----------+-----------------------------------------+
4446
4547settings.yml
4648~~~~~~~~~~~~