@@ -1,33 +1,33 @@
11.. _makefile:
22
3−========
4−Makefile
5−========
3+================
4+Makefile Targets
5+================
66
77.. _gnu-make: https://www.gnu.org/software/make/manual/make.html#Introduction
88
99.. sidebar:: build environment
1010
11− Before looking deeper at the targets, first read about :ref:`make
12− install`.
11+ Before looking deeper at the targets, first read about :ref:`make pyenv`.
1312
1413 To install system requirements follow :ref:`buildhosts`.
1514
16−All relevant build tasks are implemented in :origin:`manage.sh` and for CI or
17−IDE integration a small ``Makefile`` wrapper is available. If you are not
18−familiar with Makefiles, we recommend to read gnu-make_ introduction.
15+With the aim to simplify development cycles, started with :pull:`1756` a
16+``Makefile`` based boilerplate was added. If you are not familiar with
17+Makefiles, we recommend to read gnu-make_ introduction.
1918
2019The usage is simple, just type ``make {target-name}`` to *build* a target.
2120Calling the ``help`` target gives a first overview (``make help``):
2221
2322.. program-output:: bash -c "cd ..; make --no-print-directory help"
2423
24+
2525.. contents:: Contents
2626 :depth: 2
2727 :local:
2828 :backlinks: entry
2929
30−.. _make install:
30+.. _make pyenv:
3131
3232Python environment
3333==================
@@ -36,42 +36,31 @@ Python environment
3636
3737 ``source ./local/py3/bin/activate``
3838
39−We do no longer need to build up the virtualenv manually. Jump into your git
40−working tree and release a ``make install`` to get a virtualenv with a
41−*developer install* of searx (:origin:`setup.py`). ::
39+With Makefile we do no longer need to build up the virtualenv manually (as
40+described in the :ref:`devquickstart` guide). Jump into your git working tree
41+and release a ``make pyenv``:
42+
43+.. code:: sh
4244
4345 $ cd ~/searx-clone
44− $ make install
45− PYENV [virtualenv] installing ./requirements*.txt into local/py3
46+ $ make pyenv
47+ PYENV usage: source ./local/py3/bin/activate
4648 ...
47− PYENV OK
48− PYENV [install] pip install -e 'searx[test]'
49− ...
50− Successfully installed argparse-1.4.0 searx
51− BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
52− BUILDENV INFO:searx:Initialisation done
53− BUILDENV build utils/brand.env
5449
55−If you release ``make install`` multiple times the installation will only
56−rebuild if the sha256 sum of the *requirement files* fails. With other words:
57−the check fails if you edit the requirements listed in
58−:origin:`requirements-dev.txt` and :origin:`requirements.txt`). ::
50+With target ``pyenv`` a development environment (aka virtualenv) was build up in
51+``./local/py3/``. To make a *developer install* of searx (:origin:`setup.py`)
52+into this environment, use make target ``install``:
53+
54+.. code:: sh
5955
6056 $ make install
61− PYENV OK
62− PYENV [virtualenv] requirements.sha256 failed
63− [virtualenv] - 6cea6eb6def9e14a18bf32f8a3e... ./requirements-dev.txt
64− [virtualenv] - 471efef6c73558e391c3adb35f4... ./requirements.txt
65− ...
66− PYENV [virtualenv] installing ./requirements*.txt into local/py3
67− ...
68− PYENV OK
69− PYENV [install] pip install -e 'searx[test]'
70− ...
71− Successfully installed argparse-1.4.0 searx
72− BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
73− BUILDENV INFO:searx:Initialisation done
74− BUILDENV build utils/brand.env
57+ PYENV usage: source ./local/py3/bin/activate
58+ PYENV using virtualenv from ./local/py3
59+ PYENV install .
60+
61+You have never to think about intermediate targets like ``pyenv`` or
62+``install``, the ``Makefile`` chains them as requisites. Just run your main
63+target.
7564
7665.. sidebar:: drop environment
7766
@@ -79,7 +68,10 @@ the check fails if you edit the requirements listed in
7968 <make clean>` first.
8069
8170If you think, something goes wrong with your ./local environment or you change
82−the :origin:`setup.py` file, you have to call :ref:`make clean`.
71+the :origin:`setup.py` file (or the requirements listed in
72+:origin:`requirements-dev.txt` and :origin:`requirements.txt`), you have to call
73+:ref:`make clean`.
74+
8375
8476.. _make run:
8577
@@ -89,113 +81,117 @@ the :origin:`setup.py` file, you have to call :ref:`make clean`.
8981To get up a running a developer instance simply call ``make run``. This enables
9082*debug* option in :origin:`searx/settings.yml`, starts a ``./searx/webapp.py``
9183instance, disables *debug* option again and opens the URL in your favorite WEB
92−browser (:man:`xdg-open`)::
84+browser (:man:`xdg-open`):
9385
94− $ make run
95− PYENV OK
96− SEARX_DEBUG=1 ./manage.sh pyenv.cmd python ./searx/webapp.py
97− ...
98− INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
86+.. code:: sh
87+
88+ $ make run
89+ PYENV usage: source ./local/py3/bin/activate
90+ PYENV install .
91+ ./local/py3/bin/python ./searx/webapp.py
92+ ...
93+ INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
94+ ...
9995
10096.. _make clean:
10197
10298``make clean``
10399==============
104100
105−Drop all intermediate files, all builds, but keep sources untouched. Before
106−calling ``make clean`` stop all processes using :ref:`make install`. ::
101+Drop all intermediate files, all builds, but keep sources untouched. Includes
102+target ``pyclean`` which drops ./local environment. Before calling ``make
103+clean`` stop all processes using :ref:`make pyenv`.
104+
105+.. code:: sh
107106
108107 $ make clean
109− CLEAN pyenv
110− PYENV [virtualenv] drop ./local/py3
111− CLEAN docs -- ./build/docs ./dist/docs
112− CLEAN locally installed npm dependencies
113− CLEAN test stuff
114− CLEAN common files
108+ CLEAN pyclean
109+ CLEAN clean
115110
116111.. _make docs:
117112
118−``make docs docs.autobuild docs.clean``
119−=======================================
113+``make docs docs-live docs-clean``
114+==================================
120115
121−We describe the usage of the ``doc.*`` targets in the :ref:`How to contribute /
116+We describe the usage of the ``doc*`` targets in the :ref:`How to contribute /
122117Documentation <contrib docs>` section. If you want to edit the documentation
123−read our :ref:`make docs.live` section. If you are working in your own brand,
118+read our :ref:`make docs-live` section. If you are working in your own brand,
124119adjust your :ref:`settings global`.
125120
126−.. _make docs.gh-pages:
121+.. _make books:
127122
128−``make docs.gh-pages``
129−======================
123+``make books/{name}.html books/{name}.pdf``
124+===========================================
130125
131−To deploy on github.io first adjust your :ref:`settings global`. For any
132−further read :ref:`deploy on github.io`.
126+.. _intersphinx: https://www.sphinx-doc.org/en/stable/ext/intersphinx.html
127+.. _XeTeX: https://tug.org/xetex/
133128
134−.. _make test:
129+.. sidebar:: info
135130
136−``make test``
137−=============
131+ To build PDF a XeTeX_ is needed, see :ref:`buildhosts`.
138132
139−Runs a series of tests: :ref:`make test.pylint`, ``test.pep8``, ``test.unit``
140−and ``test.robot``. You can run tests selective, e.g.::
141133
142− $ make test.pep8 test.unit test.sh
143− TEST test.pep8 OK
144− ...
145− TEST test.unit OK
146− ...
147− TEST test.sh OK
148−
149−.. _make test.sh:
134+The ``books/{name}.*`` targets are building *books*. A *book* is a
135+sub-directory containing a ``conf.py`` file. One example is the user handbook
136+which can deployed separately (:origin:`docs/user/conf.py`). Such ``conf.py``
137+do inherit from :origin:`docs/conf.py` and overwrite values to fit *book's*
138+needs.
150139
151−``make test.sh``
152−================
140+With the help of Intersphinx_ (:ref:`reST smart ref`) the links to searx’s
141+documentation outside of the book will be bound by the object inventory of
142+``DOCS_URL``. Take into account that URLs will be picked from the inventary at
143+documentation's build time.
153144
154−:ref:`sh lint` / if you have changed some bash scripting run this test before
155−commit.
145+Use ``make docs-help`` to see which books available:
156146
157−.. _make test.pylint:
147+.. program-output:: bash -c "cd ..; make --no-print-directory docs-help"
148+ :ellipsis: 0,-6
158149
159−``make test.pylint``
160−====================
161150
162−.. _Pylint: https://www.pylint.org/
151+.. _make gh-pages:
163152
164−Pylint_ is known as one of the best source-code, bug and quality checker for the
165−Python programming language. The pylint profile we use at searx project is
166−found in project's root folder :origin:`.pylintrc`.
153+``make gh-pages``
154+=================
167155
168−.. _make search.checker:
156+To deploy on github.io first adjust your :ref:`settings global`. For any
157+further read :ref:`deploy on github.io`.
169158
170−``search.checker.{engine name}``
171−================================
159+.. _make test:
172160
173−To check all engines::
161+``make test``
162+=============
174163
175− make search.checker
164+Runs a series of tests: ``test.pep8``, ``test.unit``, ``test.robot`` and does
165+additional :ref:`pylint checks <make pylint>`. You can run tests selective,
166+e.g.:
176167
177−To check a engine with whitespace in the name like *google news* replace space
178−by underline::
168+.. code:: sh
179169
180− make search.checker.google_news
170+ $ make test.pep8 test.unit test.sh
171+ . ./local/py3/bin/activate; ./manage.sh pep8_check
172+ [!] Running pep8 check
173+ . ./local/py3/bin/activate; ./manage.sh unit_tests
174+ [!] Running unit tests
181175
182−To see HTTP requests and more use SEARX_DEBUG::
176+.. _make pylint:
183177
184− make SEARX_DEBUG=1 search.checker.google_news
178+``make pylint``
179+===============
185180
186−.. _3xx: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_redirection
181+.. _Pylint: https://www.pylint.org/
187182
188−To filter out HTTP redirects (3xx_)::
183+Before commiting its recommend to do some (more) linting. Pylint_ is known as
184+one of the best source-code, bug and quality checker for the Python programming
185+language. Pylint_ is not yet a quality gate within our searx project (like
186+:ref:`test.pep8 <make test>` it is), but Pylint_ can help to improve code
187+quality anyway. The pylint profile we use at searx project is found in
188+project's root folder :origin:`.pylintrc`.
189189
190− make SEARX_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]"
191− ...
192− Engine google news Checking
193− https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
194− https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
195− --
196− https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
197− https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
198− --
190+Code quality is a ongoing process. Don't try to fix all messages from Pylint,
191+run Pylint and check if your changed lines are bringing up new messages. If so,
192+fix it. By this, code quality gets incremental better and if there comes the
193+day, the linting is balanced out, we might decide to add Pylint as a quality
194+gate.
199195
200196
201197``make pybuild``
@@ -204,7 +200,9 @@ To filter out HTTP redirects (3xx_)::
204200.. _PyPi: https://pypi.org/
205201.. _twine: https://twine.readthedocs.io/en/latest/
206202
207−Build Python packages in ``./dist/py``::
203+Build Python packages in ``./dist/py``.
204+
205+.. code:: sh
208206
209207 $ make pybuild
210208 ...
@@ -212,11 +210,9 @@ Build Python packages in ``./dist/py``::
212210 running sdist
213211 running egg_info
214212 ...
215− running bdist_wheel
216−
217− $ ls ./dist
218− searx-0.18.0-py3-none-any.whl searx-0.18.0.tar.gz
213+ $ ls ./dist/py/
214+ searx-0.15.0-py3-none-any.whl searx-0.15.0.tar.gz
219215
220−To upload packages to PyPi_, there is also a ``pypi.upload`` target (to test use
221−``pypi.upload.test``). Since you are not the owner of :pypi:`searx` you will
222−never need to upload.
216+To upload packages to PyPi_, there is also a ``upload-pypi`` target. It needs
217+twine_ to be installed. Since you are not the owner of :pypi:`searx` you will
218+never need the latter.