git.iko.soy/searx
on master help sign in
doc: describe Makefile targets
f09459bMarkus Heiser
Makefile+4
compare with master
unifiedsplit
@@ -8,6 +8,9 @@ PYOBJECTS = searx
8DOC = docs
8DOC = docs
9PY_SETUP_EXTRAS ?= \[test\]
9PY_SETUP_EXTRAS ?= \[test\]
10
10
11+PYDIST=./dist/py
12+PYBUILD=./build/py
13+
11include utils/makefile.include
14include utils/makefile.include
12include utils/makefile.python
15include utils/makefile.python
13include utils/makefile.sphinx
16include utils/makefile.sphinx
@@ -23,6 +26,7 @@ help:
23 @echo ' install - developer install (./local)'
26 @echo ' install - developer install (./local)'
24 @echo ' uninstall - uninstall (./local)'
27 @echo ' uninstall - uninstall (./local)'
25 @echo ' gh-pages - build docs & deploy on gh-pages branch'
28 @echo ' gh-pages - build docs & deploy on gh-pages branch'
29+ @echo ' clean - drop builds and environments'
26 @echo ''
30 @echo ''
27 @$(MAKE) -s -f utils/makefile.include make-help
31 @$(MAKE) -s -f utils/makefile.include make-help
28 @echo ''
32 @echo ''
docs/conf.py+1
@@ -32,6 +32,7 @@ extlinks['origin'] = (GIT_URL + '/blob/master/%s', 'git://')
32extlinks['patch'] = (GIT_URL + '/commit/%s', '#')
32extlinks['patch'] = (GIT_URL + '/commit/%s', '#')
33extlinks['search'] = (SEARX_URL + '/%s', '#')
33extlinks['search'] = (SEARX_URL + '/%s', '#')
34extlinks['docs'] = (DOCS_URL + '/%s', 'docs: ')
34extlinks['docs'] = (DOCS_URL + '/%s', 'docs: ')
35+extlinks['pypi'] = ('https://pypi.org/project/%s', 'PyPi: ')
35
36
36extensions = [
37extensions = [
37 'sphinx.ext.extlinks',
38 'sphinx.ext.extlinks',
docs/dev/contribution_guide.rst+10−5
@@ -79,6 +79,8 @@ Translation currently takes place on :ref:`transifex <translation>`.
79 Please, do not update translation files in the repo.
79 Please, do not update translation files in the repo.
80
80
81
81
82+.. _contrib docs:
83+
82Documentation
84Documentation
83=============
85=============
84
86
@@ -91,7 +93,7 @@ Documentation
91
93
92The documentation is built using Sphinx_. So in order to be able to generate
94The documentation is built using Sphinx_. So in order to be able to generate
93the required files, you have to install it on your system. Much easier, use
95the required files, you have to install it on your system. Much easier, use
94−Makefile our targets.
96+our :ref:`makefile`.
95
97
96Here is an example which makes a complete rebuild:
98Here is an example which makes a complete rebuild:
97
99
@@ -101,6 +103,7 @@ Here is an example which makes a complete rebuild:
101 ...
103 ...
102 The HTML pages are in dist/docs.
104 The HTML pages are in dist/docs.
103
105
106+.. _make docs-live:
104
107
105live build
108live build
106----------
109----------
@@ -110,9 +113,10 @@ live build
110 It is recommended to assert a complete rebuild before deploying (use
113 It is recommended to assert a complete rebuild before deploying (use
111 ``docs-clean``).
114 ``docs-clean``).
112
115
113−Live build is like WYSIWYG, If you want to edit the documentation, its
116+Live build is like WYSIWYG. If you want to edit the documentation, its
114−recommended to use. The Makefile target ``docs-live`` builds the docs, opens URL
117+recommended to use. The Makefile target ``docs-live`` builds the docs, opens
115−in your favorite browser and rebuilds every time a reST file has been changed.
118+URL in your favorite browser and rebuilds every time a reST file has been
119+changed.
116
120
117.. code:: sh
121.. code:: sh
118
122
@@ -123,12 +127,13 @@ in your favorite browser and rebuilds every time a reST file has been changed.
123 ... Start watching changes
127 ... Start watching changes
124
128
125
129
130+.. _deploy on github.io:
126
131
127deploy on github.io
132deploy on github.io
128-------------------
133-------------------
129
134
130To deploy documentation at :docs:`github.io <.>` use Makefile target
135To deploy documentation at :docs:`github.io <.>` use Makefile target
131−``gh-pages``, which will builds the documentation, clones searx into a sub
136+:ref:`make gh-pages`, which will builds the documentation, clones searx into a sub
132folder ``gh-pages``, cleans it, copies the doc build into and runs all the
137folder ``gh-pages``, cleans it, copies the doc build into and runs all the
133needed git add, commit and push:
138needed git add, commit and push:
134
139
docs/dev/index.rst+1
@@ -11,3 +11,4 @@ Developer documentation
11 search_api
11 search_api
12 plugins
12 plugins
13 translation
13 translation
14+ makefile
docs/dev/makefile.rstnew file+217
@@ -0,0 +1,217 @@
1+.. _makefile:
2+
3+================
4+Makefile Targets
5+================
6+
7+.. sidebar:: build environment
8+
9+ Before looking deeper at the targets, first read about :ref:`makefile setup`
10+ and :ref:`make pyenv`.
11+
12+With the aim to simplify development cycles, started with :pull:`1756` a
13+``Makefile`` based boilerplate was added.
14+
15+The usage is simple, just type ``make {target-name}`` to *build* a target.
16+Calling the ``help`` target gives a first overview::
17+
18+ $ make help
19+ test - run developer tests
20+ docs - build documentation
21+ docs-live - autobuild HTML documentation while editing
22+ run - run developer instance
23+ install - developer install (./local)
24+ uninstall - uninstall (./local)
25+ gh-pages - build docs & deploy on gh-pages branch
26+ clean - drop builds and environments
27+ ...
28+
29+.. contents:: Contents
30+ :depth: 2
31+ :local:
32+ :backlinks: entry
33+
34+
35+.. _makefile setup:
36+
37+Setup
38+=====
39+
40+.. _git stash: https://git-scm.com/docs/git-stash
41+
42+The main setup is done in the :origin:`Makefile`::
43+
44+ export GIT_URL=https://github.com/asciimoo/searx
45+ export SEARX_URL=https://searx.me
46+ export DOCS_URL=https://asciimoo.github.io/searx
47+
48+.. sidebar:: fork & upstream
49+
50+ Commit changes in your (local) branch, fork or whatever, but do not push them
51+ upstream / `git stash`_ is your friend.
52+
53+:GIT_URL: Changes this, to point to your searx fork.
54+
55+:SEARX_URL: Changes this, to point to your searx instance.
56+
57+:DOCS_URL: If you host your own (branded) documentation, change this URL.
58+
59+.. _make pyenv:
60+
61+Python environment
62+==================
63+
64+.. sidebar:: activate environment
65+
66+ ``source ./local/py3/bin/activate``
67+
68+With Makefile we do no longer need to build up the virualenv manually (as
69+described in the :ref:`devquickstart` guide). Jump into your git working tree
70+and release a ``make pyenv``:
71+
72+.. code:: sh
73+
74+ $ cd ~/searx-clone
75+ $ make pyenv
76+ PYENV usage: source ./local/py3/bin/activate
77+ ...
78+
79+With target ``pyenv`` a development environment (aka virtualenv) was build up in
80+``./local/py3/``. To make a *developer install** of searx (:origin:`setup.py`)
81+into this environment make target ``install``
82+
83+.. code:: sh
84+
85+ $ make install
86+ PYENV usage: source ./local/py3/bin/activate
87+ PYENV using virtualenv from ./local/py3
88+ PYENV install .
89+
90+You have never to think about intermediate targets like ``pyenv`` or
91+``install``, the ``Makefile`` chains them as requisites. Just run your main
92+target.
93+
94+.. sidebar:: drop environment
95+
96+ To get rid of the existing environment before re-build use :ref:`clean target
97+ <make clean>` first.
98+
99+If you think, something goes wrong with your ./local environment or you change
100+the :origin:`setup.py` file (or the requirements listed in
101+:origin:`requirements-dev.txt` and :origin:`requirements.txt`), you have to call
102+:ref:`make clean`.
103+
104+
105+.. _make run:
106+
107+``make run``
108+============
109+
110+To get up a running a developer instance simply call ``make run``. This enables
111+*debug* option in :origin:`searx/settings.yml`, starts ``./searx/webapp.py``
112+instance, disables *debug* option and opens the site (xdg-open):
113+
114+.. code:: sh
115+
116+ $ make run
117+ PYENV usage: source ./local/py3/bin/activate
118+ PYENV install .
119+ ./local/py3/bin/python ./searx/webapp.py
120+ ...
121+ INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
122+ ...
123+
124+.. _make clean:
125+
126+``make clean``
127+==============
128+
129+Drop all intermediate files, all builds, but keep sources untouched. Includes
130+target ``pyclean`` which drops ./local environment. Before calling ``make
131+clean`` stop all processes using :ref:`make pyenv`.
132+
133+.. code:: sh
134+
135+ $ make clean
136+ CLEAN pyclean
137+ CLEAN clean
138+
139+.. _make docs:
140+
141+``make docs docs-live docs-clean``
142+==================================
143+
144+We describe the usage of the ``doc*`` targets in the :ref:`How to contribute /
145+Documentation <contrib docs>` section. If you want to edit the documentation
146+read our :ref:`make docs-live` section. If you are working in your own brand,
147+adjust your :ref:`Makefile setup <makefile setup>`.
148+
149+
150+.. _make gh-pages:
151+
152+``make gh-pages``
153+=================
154+
155+To deploy on github.io first adjust your :ref:`Makefile setup <makefile
156+setup>`. For any further read :ref:`deploy on github.io`.
157+
158+.. _make test:
159+
160+``make test``
161+=============
162+
163+Runs a series of tests: ``test.pep8``, ``test.unit``, ``test.robot`` and does
164+additional :ref:`pylint checks <make pylint>`. You can run tests selective,
165+e.g.:
166+
167+.. code:: sh
168+
169+ $ make test.pep8 test.unit
170+ . ./local/py3/bin/activate; ./manage.sh pep8_check
171+ [!] Running pep8 check
172+ . ./local/py3/bin/activate; ./manage.sh unit_tests
173+ [!] Running unit tests
174+
175+.. _make pylint:
176+
177+``make pylint``
178+===============
179+
180+.. _Pylint: https://www.pylint.org/
181+
182+Before commiting its recommend to do some (more) linting. Pylint_ is known as
183+one of the best source-code, bug and quality checker for the Python programming
184+language. Pylint_ is not yet a quality gate within our searx project (like
185+:ref:`test.pep8 <make test>` it is), but Pylint_ can help to improve code
186+quality anyway. The pylint profile we use at searx project is found in
187+project's root folder :origin:`.pylintrc`.
188+
189+Code quality is a ongoing process. Don't try to fix all messages from Pylint,
190+run Pylint and check if your changed lines are bringing up new messages. If so,
191+fix it. By this, code quality gets incremental better and if there comes the
192+day, the linting is balanced out, we might decide to add Pylint as a quality
193+gate.
194+
195+
196+``make pybuild``
197+================
198+
199+.. _PyPi: https://pypi.org/
200+.. _twine: https://twine.readthedocs.io/en/latest/
201+
202+Build Python packages in ``./dist/py``.
203+
204+.. code:: sh
205+
206+ $ make pybuild
207+ ...
208+ BUILD pybuild
209+ running sdist
210+ running egg_info
211+ ...
212+ $ ls ./dist/py/
213+ searx-0.15.0-py3-none-any.whl searx-0.15.0.tar.gz
214+
215+To upload packages to PyPi_, there is also a ``upload-pypi`` target. It needs
216+twine_ to be installed. Since you are not the owner of :pypi:`searx` you will
217+never need the latter.
docs/dev/quickstart.rst+24−2
@@ -4,15 +4,23 @@
4Development Quickstart
4Development Quickstart
5======================
5======================
6
6
7+.. sidebar:: :ref:`makefile`
8+
9+ For additional developer purpose there are :ref:`makefile`.
10+
7This quickstart guide gets your environment set up with searx. Furthermore, it
11This quickstart guide gets your environment set up with searx. Furthermore, it
8gives a short introduction to the ``manage.sh`` script.
12gives a short introduction to the ``manage.sh`` script.
9
13
10How to setup your development environment
14How to setup your development environment
11=========================================
15=========================================
12
16
17+.. sidebar:: :ref:`make pyenv <make pyenv>`
18+
19+ Alternatively use the :ref:`make pyenv`.
20+
13First, clone the source code of searx to the desired folder. In this case the
21First, clone the source code of searx to the desired folder. In this case the
14source is cloned to ``~/myprojects/searx``. Then create and activate the
22source is cloned to ``~/myprojects/searx``. Then create and activate the
15−searx-ve virtualenv and install the required packages using manage.sh.
23+searx-ve virtualenv and install the required packages using ``manage.sh``.
16
24
17.. code:: sh
25.. code:: sh
18
26
@@ -27,6 +35,10 @@ searx-ve virtualenv and install the required packages using manage.sh.
27How to run tests
35How to run tests
28================
36================
29
37
38+.. sidebar:: :ref:`make test.unit <make test>`
39+
40+ Alternatively use the ``test.pep8``, ``test.unit``, ``test.robot`` targets.
41+
30Tests can be run using the ``manage.sh`` script. Following tests and checks are
42Tests can be run using the ``manage.sh`` script. Following tests and checks are
31available:
43available:
32
44
@@ -41,7 +53,8 @@ For example unit tests are run with the command below:
41
53
42 ./manage.sh unit_tests
54 ./manage.sh unit_tests
43
55
44−For further test options, please consult the help of the ``manage.sh`` script.
56+For further test options, please consult the help of the ``manage.sh`` script or
57+read :ref:`make test`.
45
58
46
59
47How to compile styles and javascript
60How to compile styles and javascript
@@ -97,6 +110,11 @@ After installing grunt, the files can be built using the following command:
97Tips for debugging/development
110Tips for debugging/development
98==============================
111==============================
99
112
113+.. sidebar:: :ref:`make run`
114+
115+ Makefile target ``run`` already enables debug option for your developer
116+ session / see :ref:`make run`.
117+
100Turn on debug logging
118Turn on debug logging
101 Whether you are working on a new engine or trying to eliminate a bug, it is
119 Whether you are working on a new engine or trying to eliminate a bug, it is
102 always a good idea to turn on debug logging. When debug logging is enabled a
120 always a good idea to turn on debug logging. When debug logging is enabled a
@@ -104,6 +122,10 @@ Turn on debug logging
104 message. It can be turned on by setting ``debug: False`` to ``debug: True`` in
122 message. It can be turned on by setting ``debug: False`` to ``debug: True`` in
105 :origin:`settings.yml <searx/settings.yml>`.
123 :origin:`settings.yml <searx/settings.yml>`.
106
124
125+.. sidebar:: :ref:`make test`
126+
127+ Alternatively use the :ref:`make test` targets.
128+
107Run ``./manage.sh tests`` before creating a PR.
129Run ``./manage.sh tests`` before creating a PR.
108 Failing build on Travis is common because of PEP8 checks. So a new commit
130 Failing build on Travis is common because of PEP8 checks. So a new commit
109 must be created containing these format fixes. This phase can be skipped if
131 must be created containing these format fixes. This phase can be skipped if