@@ -1,33 +1,33 @@
11.. _makefile:
22
3−================
4−Makefile Targets
5−================
3+========
4+Makefile
5+========
66
77.. _gnu-make: https://www.gnu.org/software/make/manual/make.html#Introduction
88
99.. sidebar:: build environment
1010
11− Before looking deeper at the targets, first read about :ref:`make pyenv`.
11+ Before looking deeper at the targets, first read about :ref:`make
12+ install`.
1213
1314 To install system requirements follow :ref:`buildhosts`.
1415
15−With the aim to simplify development cycles, started with :pull:`1756` a
16−``Makefile`` based boilerplate was added. If you are not familiar with
17−Makefiles, we recommend to read gnu-make_ introduction.
16+All relevant build tasks are implemented in :origin:`manage.sh` and for CI or
17+IDE integration a small ``Makefile`` wrapper is available. If you are not
18+familiar with Makefiles, we recommend to read gnu-make_ introduction.
1819
1920The usage is simple, just type ``make {target-name}`` to *build* a target.
2021Calling the ``help`` target gives a first overview (``make help``):
2122
2223.. program-output:: bash -c "cd ..; make --no-print-directory help"
2324
24−
2525.. contents:: Contents
2626 :depth: 2
2727 :local:
2828 :backlinks: entry
2929
30−.. _make pyenv:
30+.. _make install:
3131
3232Python environment
3333==================
@@ -36,31 +36,42 @@ Python environment
3636
3737 ``source ./local/py3/bin/activate``
3838
39−With Makefile we do no longer need to build up the virtualenv manually (as
40−described in the :ref:`devquickstart` guide). Jump into your git working tree
41−and release a ``make pyenv``:
42−
43−.. code:: sh
39+We do no longer need to build up the virtualenv manually. Jump into your git
40+working tree and release a ``make install`` to get a virtualenv with a
41+*developer install* of searx (:origin:`setup.py`). ::
4442
4543 $ cd ~/searx-clone
46− $ make pyenv
47− PYENV usage: source ./local/py3/bin/activate
44+ $ make install
45+ PYENV [virtualenv] installing ./requirements*.txt into local/py3
4846 ...
47+ PYENV OK
48+ PYENV [install] pip install -e 'searx[test]'
49+ ...
50+ Successfully installed argparse-1.4.0 searx
51+ BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
52+ BUILDENV INFO:searx:Initialisation done
53+ BUILDENV build utils/brand.env
4954
50−With target ``pyenv`` a development environment (aka virtualenv) was build up in
51−``./local/py3/``. To make a *developer install* of searx (:origin:`setup.py`)
52−into this environment, use make target ``install``:
53−
54−.. code:: sh
55+If you release ``make install`` multiple times the installation will only
56+rebuild if the sha256 sum of the *requirement files* fails. With other words:
57+the check fails if you edit the requirements listed in
58+:origin:`requirements-dev.txt` and :origin:`requirements.txt`). ::
5559
5660 $ make install
57− PYENV usage: source ./local/py3/bin/activate
58− PYENV using virtualenv from ./local/py3
59− PYENV install .
60−
61−You have never to think about intermediate targets like ``pyenv`` or
62−``install``, the ``Makefile`` chains them as requisites. Just run your main
63−target.
61+ PYENV OK
62+ PYENV [virtualenv] requirements.sha256 failed
63+ [virtualenv] - 6cea6eb6def9e14a18bf32f8a3e... ./requirements-dev.txt
64+ [virtualenv] - 471efef6c73558e391c3adb35f4... ./requirements.txt
65+ ...
66+ PYENV [virtualenv] installing ./requirements*.txt into local/py3
67+ ...
68+ PYENV OK
69+ PYENV [install] pip install -e 'searx[test]'
70+ ...
71+ Successfully installed argparse-1.4.0 searx
72+ BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
73+ BUILDENV INFO:searx:Initialisation done
74+ BUILDENV build utils/brand.env
6475
6576.. sidebar:: drop environment
6677
@@ -68,10 +79,7 @@ target.
6879 <make clean>` first.
6980
7081If you think, something goes wrong with your ./local environment or you change
71−the :origin:`setup.py` file (or the requirements listed in
72−:origin:`requirements-dev.txt` and :origin:`requirements.txt`), you have to call
73−:ref:`make clean`.
74−
82+the :origin:`setup.py` file, you have to call :ref:`make clean`.
7583
7684.. _make run:
7785
@@ -81,117 +89,113 @@ the :origin:`setup.py` file (or the requirements listed in
8189To get up a running a developer instance simply call ``make run``. This enables
8290*debug* option in :origin:`searx/settings.yml`, starts a ``./searx/webapp.py``
8391instance, disables *debug* option again and opens the URL in your favorite WEB
84−browser (:man:`xdg-open`):
85−
86−.. code:: sh
92+browser (:man:`xdg-open`)::
8793
88− $ make run
89− PYENV usage: source ./local/py3/bin/activate
90− PYENV install .
91− ./local/py3/bin/python ./searx/webapp.py
92− ...
93− INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
94− ...
94+ $ make run
95+ PYENV OK
96+ SEARX_DEBUG=1 ./manage.sh pyenv.cmd python ./searx/webapp.py
97+ ...
98+ INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
9599
96100.. _make clean:
97101
98102``make clean``
99103==============
100104
101−Drop all intermediate files, all builds, but keep sources untouched. Includes
102−target ``pyclean`` which drops ./local environment. Before calling ``make
103−clean`` stop all processes using :ref:`make pyenv`.
104−
105−.. code:: sh
105+Drop all intermediate files, all builds, but keep sources untouched. Before
106+calling ``make clean`` stop all processes using :ref:`make install`. ::
106107
107108 $ make clean
108− CLEAN pyclean
109− CLEAN clean
109+ CLEAN pyenv
110+ PYENV [virtualenv] drop ./local/py3
111+ CLEAN docs -- ./build/docs ./dist/docs
112+ CLEAN locally installed npm dependencies
113+ CLEAN test stuff
114+ CLEAN common files
110115
111116.. _make docs:
112117
113−``make docs docs-live docs-clean``
114−==================================
118+``make docs docs.autobuild docs.clean``
119+=======================================
115120
116−We describe the usage of the ``doc*`` targets in the :ref:`How to contribute /
121+We describe the usage of the ``doc.*`` targets in the :ref:`How to contribute /
117122Documentation <contrib docs>` section. If you want to edit the documentation
118−read our :ref:`make docs-live` section. If you are working in your own brand,
123+read our :ref:`make docs.live` section. If you are working in your own brand,
119124adjust your :ref:`settings global`.
120125
121−.. _make books:
126+.. _make docs.gh-pages:
122127
123−``make books/{name}.html books/{name}.pdf``
124−===========================================
128+``make docs.gh-pages``
129+======================
125130
126−.. _intersphinx: https://www.sphinx-doc.org/en/stable/ext/intersphinx.html
127−.. _XeTeX: https://tug.org/xetex/
131+To deploy on github.io first adjust your :ref:`settings global`. For any
132+further read :ref:`deploy on github.io`.
128133
129−.. sidebar:: info
134+.. _make test:
130135
131− To build PDF a XeTeX_ is needed, see :ref:`buildhosts`.
136+``make test``
137+=============
132138
139+Runs a series of tests: :ref:`make test.pylint`, ``test.pep8``, ``test.unit``
140+and ``test.robot``. You can run tests selective, e.g.::
133141
134−The ``books/{name}.*`` targets are building *books*. A *book* is a
135−sub-directory containing a ``conf.py`` file. One example is the user handbook
136−which can deployed separately (:origin:`docs/user/conf.py`). Such ``conf.py``
137−do inherit from :origin:`docs/conf.py` and overwrite values to fit *book's*
138−needs.
142+ $ make test.pep8 test.unit test.sh
143+ TEST test.pep8 OK
144+ ...
145+ TEST test.unit OK
146+ ...
147+ TEST test.sh OK
139148
140−With the help of Intersphinx_ (:ref:`reST smart ref`) the links to searx’s
141−documentation outside of the book will be bound by the object inventory of
142−``DOCS_URL``. Take into account that URLs will be picked from the inventary at
143−documentation's build time.
149+.. _make test.sh:
144150
145−Use ``make docs-help`` to see which books available:
151+``make test.sh``
152+================
146153
147−.. program-output:: bash -c "cd ..; make --no-print-directory docs-help"
148− :ellipsis: 0,-6
154+:ref:`sh lint` / if you have changed some bash scripting run this test before
155+commit.
149156
157+.. _make test.pylint:
150158
151−.. _make gh-pages:
159+``make test.pylint``
160+====================
152161
153−``make gh-pages``
154−=================
162+.. _Pylint: https://www.pylint.org/
155163
156−To deploy on github.io first adjust your :ref:`settings global`. For any
157−further read :ref:`deploy on github.io`.
164+Pylint_ is known as one of the best source-code, bug and quality checker for the
165+Python programming language. The pylint profile we use at searx project is
166+found in project's root folder :origin:`.pylintrc`.
158167
159−.. _make test:
168+.. _make search.checker:
160169
161−``make test``
162−=============
170+``search.checker.{engine name}``
171+================================
163172
164−Runs a series of tests: ``test.pep8``, ``test.unit``, ``test.robot`` and does
165−additional :ref:`pylint checks <make pylint>`. You can run tests selective,
166−e.g.:
173+To check all engines::
167174
168−.. code:: sh
175+ make search.checker
169176
170− $ make test.pep8 test.unit test.sh
171− . ./local/py3/bin/activate; ./manage.sh pep8_check
172− [!] Running pep8 check
173− . ./local/py3/bin/activate; ./manage.sh unit_tests
174− [!] Running unit tests
177+To check a engine with whitespace in the name like *google news* replace space
178+by underline::
175179
176−.. _make pylint:
180+ make search.checker.google_news
177181
178−``make pylint``
179−===============
182+To see HTTP requests and more use SEARX_DEBUG::
180183
181−.. _Pylint: https://www.pylint.org/
184+ make SEARX_DEBUG=1 search.checker.google_news
185+
186+.. _3xx: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_redirection
182187
183−Before commiting its recommend to do some (more) linting. Pylint_ is known as
184−one of the best source-code, bug and quality checker for the Python programming
185−language. Pylint_ is not yet a quality gate within our searx project (like
186−:ref:`test.pep8 <make test>` it is), but Pylint_ can help to improve code
187−quality anyway. The pylint profile we use at searx project is found in
188−project's root folder :origin:`.pylintrc`.
188+To filter out HTTP redirects (3xx_)::
189189
190−Code quality is a ongoing process. Don't try to fix all messages from Pylint,
191−run Pylint and check if your changed lines are bringing up new messages. If so,
192−fix it. By this, code quality gets incremental better and if there comes the
193−day, the linting is balanced out, we might decide to add Pylint as a quality
194−gate.
190+ make SEARX_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]"
191+ ...
192+ Engine google news Checking
193+ https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
194+ https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
195+ --
196+ https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
197+ https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
198+ --
195199
196200
197201``make pybuild``
@@ -200,9 +204,7 @@ gate.
200204.. _PyPi: https://pypi.org/
201205.. _twine: https://twine.readthedocs.io/en/latest/
202206
203−Build Python packages in ``./dist/py``.
204−
205−.. code:: sh
207+Build Python packages in ``./dist/py``::
206208
207209 $ make pybuild
208210 ...
@@ -210,9 +212,11 @@ Build Python packages in ``./dist/py``.
210212 running sdist
211213 running egg_info
212214 ...
213− $ ls ./dist/py/
214− searx-0.15.0-py3-none-any.whl searx-0.15.0.tar.gz
215+ running bdist_wheel
216+
217+ $ ls ./dist
218+ searx-0.18.0-py3-none-any.whl searx-0.18.0.tar.gz
215219
216−To upload packages to PyPi_, there is also a ``upload-pypi`` target. It needs
217−twine_ to be installed. Since you are not the owner of :pypi:`searx` you will
218−never need the latter.
220+To upload packages to PyPi_, there is also a ``pypi.upload`` target (to test use
221+``pypi.upload.test``). Since you are not the owner of :pypi:`searx` you will
222+never need to upload.