on master help sign in
docs: improve description of uwsgi & ngingx setup

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

commit387c6a77691fec514704bdf178b9ab94ad4abb40 authorMarkus Heiser <markus.heiser@darmarit.de> dateFri 6 Mar 2020, 14:47 (7 years ago) parentcbc08fd
11 files changed+162−63
docs/admin/filtron.rst+13−7 docs/admin/installation-nginx.rst+103−46 docs/admin/installation-searx.rst+2 docs/admin/installation-uwsgi.rst+2 docs/admin/morty.rst+1−1 docs/utils/filtron.sh.rst+2−1 docs/utils/morty.sh.rst+1−1 utils/lib.sh+1 utils/searx.sh+3−1 utils/templates/etc/uwsgi/apps-archlinux/searx.ini+17−3 utils/templates/etc/uwsgi/apps-available/searx.ini+17−3
docs/admin/filtron.rst+13−7
@@ -1,5 +1,5 @@
11
2−.. _searx_filtron:
2+.. _searx filtron:
33
44==========================
55How to protect an instance
@@ -8,6 +8,8 @@ How to protect an instance
88.. sidebar:: further reading
99
1010 - :ref:`filtron.sh`
11+ - :ref:`nginx searx site`
12+
1113
1214.. contents:: Contents
1315 :depth: 2
@@ -150,6 +152,8 @@ of:
150152 ]
151153
152154
155+.. _filtron route request:
156+
153157Route request through filtron
154158=============================
155159
@@ -167,12 +171,14 @@ Use it along with ``nginx`` with the following example configuration.
167171.. code:: nginx
168172
169173 location / {
170− proxy_set_header Host $http_host;
171− proxy_set_header X-Real-IP $remote_addr;
172− proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
173− proxy_set_header X-Scheme $scheme;
174− proxy_pass http://127.0.0.1:4004/;
174+ proxy_pass http://127.0.0.1:4004/;
175+
176+ proxy_set_header Host $http_host;
177+ proxy_set_header X-Real-IP $remote_addr;
178+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
179+ proxy_set_header X-Scheme $scheme;
175180 }
176181
177182Requests are coming from port 4004 going through filtron and then forwarded to
178−port 8888 where a searx is being run.
183+port 8888 where a searx is being run. For a complete setup see: :ref:`nginx
184+searx site`.
docs/admin/installation-nginx.rst+103−46
@@ -12,6 +12,12 @@ Install with nginx
1212 http://nginx.org/en/docs/beginners_guide.html
1313.. _Getting Started wiki:
1414 https://www.nginx.com/resources/wiki/start/
15+.. _uWSGI support from nginx:
16+ https://uwsgi-docs.readthedocs.io/en/latest/Nginx.html
17+.. _uwsgi_params:
18+ https://uwsgi-docs.readthedocs.io/en/latest/Nginx.html#configuring-nginx
19+.. _SCRIPT_NAME:
20+ https://werkzeug.palletsprojects.com/en/1.0.x/wsgi/#werkzeug.wsgi.get_script_name
1521
1622.. contents:: Contents
1723 :depth: 2
@@ -98,8 +104,8 @@ see a *Fedora Webserver - Test Page*. The test page comes from the default
98104
99105.. _nginx searx site:
100106
101−A searx site
102−============
107+A nginx searx site
108+==================
103109
104110.. sidebar:: public to the internet?
105111
@@ -134,33 +140,42 @@ Started wiki`_ is always a good resource *to keep in the pocket*.
134140
135141.. tabs::
136142
143+ .. group-tab:: searx via filtron plus morty
137144
138− .. group-tab:: filtron at ``/`` & ``/morty``
139−
140− Use this setup, if your instance is public to the internet:
145+ Use this setup, if your instance is public to the internet, compare
146+ figure: :ref:`architecture <arch public>`. Configure a reverse proxy for
147+ :ref:`filtron <filtron.sh>`, listening on *localhost 4004* (:ref:`filtron
148+ route request`):
141149
142150 .. code:: nginx
143151
144152 location / {
145− proxy_set_header Host $http_host;
146− proxy_set_header X-Real-IP $remote_addr;
147− proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
148− proxy_set_header X-Scheme $scheme;
149153 proxy_pass http://127.0.0.1:4004/;
154+
155+ proxy_set_header Host $http_host;
156+ proxy_set_header X-Real-IP $remote_addr;
157+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
158+ proxy_set_header X-Scheme $scheme;
150159 }
151160
161+
162+ Configure reverse proxy for :ref:`morty <searx morty>`, listening on
163+ *localhost 3000*:
164+
152165 .. code:: nginx
153166
154167 location /morty {
155− proxy_set_header Host $http_host;
156− proxy_set_header X-Real-IP $remote_addr;
157− proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
158− proxy_set_header X-Scheme $scheme;
159168 proxy_pass http://127.0.0.1:3000/;
169+
170+ proxy_set_header Host $http_host;
171+ proxy_set_header X-Real-IP $remote_addr;
172+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
173+ proxy_set_header X-Scheme $scheme;
160174 }
161175
162− For a fully result proxification add :ref:`morty's <searx_morty>` public
163− URL to your :origin:`searx/settings.yml`:
176+ Note that reverse proxy advised to be used in case of single-user or
177+ low-traffic instances. For a fully result proxification add :ref:`morty's
178+ <searx morty>` **public URL** to your :origin:`searx/settings.yml`:
164179
165180 .. code:: yaml
166181
@@ -169,66 +184,108 @@ Started wiki`_ is always a good resource *to keep in the pocket*.
169184 url : http://searx.example.com/
170185
171186
172− .. group-tab:: searx at ``/``
187+ .. group-tab:: proxy or uWSGI
173188
174− Use this setup only, if your instance is **NOT** public to the internet:
189+ Be warned, with this setup, your Instance isn't :ref:`protected <searx
190+ filtron>`. Nevertheless it is good enough for intranet usage and it is a
191+ excellent example of; *how different services can be set up*. The next
192+ example shows a reverse proxy configuration wrapping the :ref:`searx-uWSGI
193+ application <uwsgi configuration>`, listening on ``http =
194+ 127.0.0.1:8888``.
175195
176196 .. code:: nginx
177197
178− server {
179− listen 80;
180− listen [::]:80;
198+ location / {
199+ proxy_pass http://127.0.0.1:8888;
181200
201+ proxy_set_header Host $host;
202+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
203+ proxy_set_header X-Scheme $scheme;
204+ proxy_set_header X-Script-Name /searx;
205+ proxy_buffering off;
206+ }
207+
208+ Alternatively you can use the `uWSGI support from nginx`_ via unix
209+ sockets. For socket communication, you have to activate ``socket =
210+ /run/uwsgi/app/searx/socket`` and comment out the ``http =
211+ 127.0.0.1:8888`` configuration in your :ref:`uwsgi ini file <uwsgi
212+ configuration>`.
213+
214+ The example shows a nginx virtual ``server`` configuration, listening on
215+ port 80 (IPv4 and IPv6 http://[::]:80). The uWSGI app is configured at
216+ location ``/`` by importing the `uwsgi_params`_ and passing requests to
217+ the uWSGI socket (``uwsgi_pass``). The ``server``\'s root points to the
218+ :ref:`searx-src clone <searx-src>` and wraps directly the
219+ :origin:`searx/static/` content at ``location /static``.
220+
221+ .. code:: nginx
222+
223+ server {
182224 # replace searx.example.com with your server's public name
183225 server_name searx.example.com;
184226
185− root /usr/local/searx/searx;
186−
187− location /static {
188− }
227+ listen 80;
228+ listen [::]:80;
189229
190230 location / {
191231 include uwsgi_params;
192232 uwsgi_pass unix:/run/uwsgi/app/searx/socket;
193233 }
234+
235+ root /usr/local/searx/searx-src/searx;
236+ location /static { }
194237 }
195238
196− .. group-tab:: searx at ``/searx``
239+ If not already exists, create a folder for the unix sockets, which can be
240+ used by the searx account:
197241
198− Use this setup only, if your instance is **NOT** public to the internet:
242+ .. code:: bash
199243
200− .. code:: nginx
244+ mkdir -p /run/uwsgi/app/searx/
245+ sudo -H chown -R searx:searx /run/uwsgi/app/searx/
201246
202− location /searx/static {
203− alias /usr/local/searx/searx/static;
204− }
247+ .. group-tab:: subdirectory URL
248+
249+ Be warned, with these setups, your Instance isn't :ref:`protected <searx
250+ filtron>`. The examples are just here to demonstrate how to export the
251+ searx application from a subdirectory URL
252+ http://searx.example.com/searx/\.
253+
254+ .. code:: nginx
205255
206256 location /searx {
207− uwsgi_param SCRIPT_NAME /searx;
208− include uwsgi_params;
209− uwsgi_pass unix:/run/uwsgi/app/searx/socket;
257+ proxy_pass http://127.0.0.1:8888;
258+
259+ proxy_set_header Host $host;
260+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
261+ proxy_set_header X-Scheme $scheme;
262+ proxy_set_header X-Script-Name /searx;
263+ proxy_buffering off;
210264 }
211265
266+ location /searx/static {
267+ alias /usr/local/searx/searx-src/searx/static;
268+ }
212269
213− **OR** using reverse proxy. Please, note that reverse proxy advised to be
214− used in case of single-user or low-traffic instances.
270+ The ``X-Script-Name /searx`` is needed by the searx implementation to
271+ calculate relative URLs correct. The next example shows a uWSGI
272+ configuration. Since there are no HTTP headers in a (u)WSGI protocol, the
273+ value is shipped via the SCRIPT_NAME_ in the WSGI environment.
215274
216275 .. code:: nginx
217276
218277 location /searx/static {
219− alias /usr/local/searx/searx/static;
278+ alias /usr/local/searx/searx-src/searx;
220279 }
221280
222281 location /searx {
223− proxy_pass http://127.0.0.1:8888;
224− proxy_set_header Host $host;
225− proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
226− proxy_set_header X-Scheme $scheme;
227− proxy_set_header X-Script-Name /searx;
228− proxy_buffering off;
282+ uwsgi_param SCRIPT_NAME /searx;
283+ include uwsgi_params;
284+ uwsgi_pass unix:/run/uwsgi/app/searx/socket;
229285 }
230286
231− Enable ``base_url`` in :origin:`searx/settings.yml`
287+ For searx to work correctly the ``base_url`` must be set in the
288+ :origin:`searx/settings.yml`.
232289
233290 .. code:: yaml
234291
@@ -246,21 +303,21 @@ Restart service:
246303 .. code:: sh
247304
248305 sudo -H systemctl restart nginx
249− sudo -H systemctl restart uwsgi
306+ sudo -H service uwsgi restart searx
250307
251308 .. group-tab:: Arch Linux
252309
253310 .. code:: sh
254311
255312 sudo -H systemctl restart nginx
256− sudo -H systemctl restart uwsgi
313+ sudo -H systemctl restart uwsgi@searx
257314
258315 .. group-tab:: Fedora
259316
260317 .. code:: sh
261318
262319 sudo -H systemctl restart nginx
263− sudo -H systemctl restart uwsgi
320+ sudo -H touch /etc/uwsgi.d/searx.ini
264321
265322
266323Disable logs
docs/admin/installation-searx.rst+2
@@ -32,6 +32,8 @@ Create user
3232 :start-after: START create user
3333 :end-before: END create user
3434
35+.. _searx-src:
36+
3537install searx & dependencies
3638============================
3739
docs/admin/installation-uwsgi.rst+2
@@ -92,6 +92,8 @@ could control specific instance(s) by issuing::
9292
9393My experience is, that this command is a bit buggy.
9494
95+.. _uwsgi configuration:
96+
9597Alltogether
9698===========
9799
docs/admin/morty.rst+1−1
@@ -1,5 +1,5 @@
11
2−.. _searx_morty:
2+.. _searx morty:
33
44=========================
55How to setup result proxy
docs/utils/filtron.sh.rst+2−1
@@ -8,7 +8,7 @@
88.. sidebar:: further reading
99
1010 - :ref:`installation`
11− - :ref:`searx_filtron`
11+ - :ref:`searx filtron`
1212 - :ref:`architecture`
1313
1414.. _Go: https://golang.org/
@@ -64,6 +64,7 @@ To install searx in your public HTTP server use:
6464 $ sudo -H a2enmod proxy
6565 $ sudo -H a2enmod proxy_http
6666
67+.. _filtron.sh overview:
6768
6869Overview
6970========
docs/utils/morty.sh.rst+1−1
@@ -26,7 +26,7 @@ into this user account.
2626
2727.. hint::
2828
29− To add morty to your searx instance read chapter :reF:`searx_morty`.
29+ To add morty to your searx instance read chapter :ref:`searx morty`.
3030
3131
3232Overview
utils/lib.sh+1
@@ -881,6 +881,7 @@ uWSGI_enable_app() {
881881 mkdir -p "${uWSGI_APPS_ENABLED}"
882882 rm -f "${uWSGI_APPS_ENABLED}/${CONF}"
883883 ln -s "${uWSGI_APPS_AVAILABLE}/${CONF}" "${uWSGI_APPS_ENABLED}/${CONF}"
884+ systemctl enable "uwsgi@${CONF%.*}"
884885 info_msg "enabled uWSGI app: ${CONF} (restart required)"
885886 ;;
886887 fedora-*)
utils/searx.sh+3−1
@@ -88,7 +88,7 @@ usage() {
8888usage::
8989
9090 $(basename "$0") shell
91− $(basename "$0") install [all|user|searx-src|pyenv|apache]
91+ $(basename "$0") install [all|user|searx-src|pyenv|uwsgi|apache]
9292 $(basename "$0") update [searx]
9393 $(basename "$0") remove [all|user|pyenv|searx-src]
9494 $(basename "$0") activate [service]
@@ -104,6 +104,7 @@ install / remove
104104 :user: add/remove service user '$SERVICE_USER' ($SERVICE_HOME)
105105 :searx-src: clone $SEARX_GIT_URL
106106 :pyenv: create/remove virtualenv (python) in $SEARX_PYENV
107+ :uwsgi: install searx uWSGI application
107108 :settings: reinstall settings from ${REPO_ROOT}/searx/settings.yml
108109update searx
109110 Update searx installation ($SERVICE_HOME)
@@ -165,6 +166,7 @@ main() {
165166 pyenv) create_pyenv ;;
166167 searx-src) clone_searx ;;
167168 settings) install_settings ;;
169+ uwsgi) install_searx_uwsgi;;
168170 *) usage "$_usage"; exit 42;;
169171 esac ;;
170172 update)
utils/templates/etc/uwsgi/apps-archlinux/searx.ini+17−3
@@ -57,10 +57,24 @@ virtualenv = ${SEARX_PYENV}
5757pythonpath = ${SEARX_SRC}
5858
5959
60−# plugin http
61−# -----------
60+# speak to upstream
61+# -----------------
6262#
63−# https://uwsgi-docs.readthedocs.io/en/latest/Options.html#plugin-http
63+# Activate the 'http' configuration for filtron or activate the 'socket'
64+# configuration if you setup your HTTP server to use uWSGI protocol via sockets.
6465
66+# using IP:
67+#
68+# https://uwsgi-docs.readthedocs.io/en/latest/Options.html#plugin-http
6569# Native HTTP support: https://uwsgi-docs.readthedocs.io/en/latest/HTTP.html
70+
6671http = ${SEARX_INTERNAL_URL}
72+
73+# using unix-sockets:
74+#
75+# On some distributions you need to create the app folder for the sockets::
76+#
77+# mkdir -p /run/uwsgi/app/searx/socket
78+# chmod -R ${SERVICE_USER}:${SERVICE_GROUP} /run/uwsgi/app/searx/socket
79+#
80+# socket = /run/uwsgi/app/searx/socket
utils/templates/etc/uwsgi/apps-available/searx.ini+17−3
@@ -56,10 +56,24 @@ virtualenv = ${SEARX_PYENV}
5656pythonpath = ${SEARX_SRC}
5757
5858
59−# plugin http
60−# -----------
59+# speak to upstream
60+# -----------------
6161#
62−# https://uwsgi-docs.readthedocs.io/en/latest/Options.html#plugin-http
62+# Activate the 'http' configuration for filtron or activate the 'socket'
63+# configuration if you setup your HTTP server to use uWSGI protocol via sockets.
6364
65+# using IP:
66+#
67+# https://uwsgi-docs.readthedocs.io/en/latest/Options.html#plugin-http
6468# Native HTTP support: https://uwsgi-docs.readthedocs.io/en/latest/HTTP.html
69+
6570http = ${SEARX_INTERNAL_URL}
71+
72+# using unix-sockets:
73+#
74+# On some distributions you need to create the app folder for the sockets::
75+#
76+# mkdir -p /run/uwsgi/app/searx/socket
77+# chmod -R ${SERVICE_USER}:${SERVICE_GROUP} /run/uwsgi/app/searx/socket
78+#
79+# socket = /run/uwsgi/app/searx/socket