733 files changed+96824−99331
.config.sh+5−1
@@ -26,8 +26,12 @@ fi
2626# ---------
2727
2828# SEARX_INTERNAL_URL="127.0.0.1:8888"
29+# SEARX_SETTINGS_TEMPLATE="${REPO_ROOT}/utils/templates/etc/searx/use_default_settings.yml"
2930
30−# Only change, if you maintain a searx brand in your searx fork.
31+# Only change, if you maintain a searx brand in your searx fork (GIT_URL) which
32+# is not hold by branch 'master'. The branch has to be a local branch, in the
33+# repository from which you install (which is most often the case). If you want
34+# to install branch 'foo', don't forget to run 'git branch foo origin/foo' once.
3135# GIT_BRANCH="${GIT_BRANCH:-master}"
3236
3337# filtron.sh
.coveragerc+1−10
@@ -1,15 +1,6 @@
11[run]
22branch = True
3−source =
4− searx/engines
5− searx/__init__.py
6− searx/autocomplete.py
7− searx/https_rewrite.py
8− searx/languages.py
9− searx/search.py
10− searx/testing.py
11− searx/utils.py
12− searx/webapp.py
3+source = searx
134
145[report]
156show_missing = True
.dir-locals.el+1−1
@@ -25,7 +25,7 @@
2525;; Alternatively create the virtualenv, source it and install jedi + epc
2626;; (required by `emacs-jedi <https://tkf.github.io/emacs-jedi>`_)::
2727;;
28−;; $ virtualenv --python=python3 "--no-site-packages" ./local/py3
28+;; $ python -m venv ./local/py3
2929;; ...
3030;; $ source ./local/py3/bin/activate
3131;; (py3)$ # now install into the activated 'py3' environment ..
.github/ISSUE_TEMPLATE/bug-report.mdnew file+39
@@ -0,0 +1,39 @@
1+---
2+name: Bug report
3+about: Report a bug in Searx
4+title: ''
5+labels: bug
6+assignees: ''
7+
8+---
9+<!-- PLEASE FILL THESE FIELDS, IT REALLY HELPS THE MAINTAINERS OF SEARX -->
10+
11+**Version of Searx, commit number if you are using on master branch and stipulate if you forked Searx**
12+<!-- If you are running on master branch using git execute this command
13+in order to fetch the latest commit ID:
14+```
15+git log -1
16+```
17+If you are using searx-docker then look at the bottom of the Searx page
18+and check for the version after "Powered by searx"
19+
20+Please also stipulate if you are using a forked version of Searx and
21+include a link to the fork source code.
22+-->
23+**How did you install Searx?**
24+<!-- Did you install Searx using the official wiki or using searx-docker
25+or manually by executing the searx/webapp.py file? -->
26+**What happened?**
27+<!-- A clear and concise description of what the bug is. -->
28+
29+**How To Reproduce**
30+<!-- How can we reproduce this issue? (as minimally and as precisely as possible) -->
31+
32+**Expected behavior**
33+<!-- A clear and concise description of what you expected to happen. -->
34+
35+**Screenshots & Logs**
36+<!-- If applicable, add screenshots, logs to help explain your problem. -->
37+
38+**Additional context**
39+<!-- Add any other context about the problem here. -->
.github/ISSUE_TEMPLATE/config.ymlnew file+14
@@ -0,0 +1,14 @@
1+blank_issues_enabled: true
2+contact_links:
3+ - name: Installation issue using the official wiki
4+ url: https://github.com/searx/searx/discussions/new?category_id=32001257
5+ about: Ask for help if you are having some issue with installing Searx using the official wiki.
6+ - name: Installation issue using searx-docker
7+ url: https://github.com/searx/searx/discussions/new?category_id=32001259
8+ about: Ask for help if you are having some issue with installing Searx using searx-docker.
9+ - name: Installation issue without using the official wiki nor searx-docker
10+ url: https://github.com/searx/searx/discussions/new?category_id=32001260
11+ about: Ask for help if you are having some issue with installing Searx manually without using the wiki nor searx-docker.
12+ - name: Report a bug on a public Searx instance
13+ url: https://github.com/searx/searx/discussions/new?category_id=32001412
14+ about: Report a bug that you discovered on a public Searx instance.
.github/ISSUE_TEMPLATE/engine-request.mdnew file+31
@@ -0,0 +1,31 @@
1+---
2+name: Engine request
3+about: Request a new engine in Searx
4+title: ''
5+labels: enhancement, engine request
6+assignees: ''
7+
8+---
9+<!-- PLEASE FILL THESE FIELDS, IT REALLY HELPS THE MAINTAINERS OF SEARX -->
10+
11+**Working URL to the engine**
12+<!-- Please check if the engine is responding correctly before submitting it. -->
13+
14+**Why do you want to add this engine?**
15+<!-- What's special about this engine? Is it open source or libre? -->
16+
17+**Features of this engine**
18+<!-- Features of this engine: Doesn't track its users, fast, easy to integrate, ... -->
19+
20+**How can Searx fetch the information from this engine?**
21+<!-- List API URL, example code (using the correct markdown) and more
22+that could be useful for the developers in order to implement this engine.
23+If you don't know what to write, let this part blank.>
24+
25+**Applicable category of this engine**
26+<!-- Where should this new engine fit in Searx? Current categories in Searx:
27+general, files, images, it, map, music, news, science, social media and videos.
28+You can add multiple categories at the same time. -->
29+
30+**Additional context**
31+<!-- Add any other context about this engine here. -->
.github/ISSUE_TEMPLATE/feature-request.mdnew file+21
@@ -0,0 +1,21 @@
1+---
2+name: Feature request
3+about: Request a new feature in Searx
4+title: ''
5+labels: enhancement
6+assignees: ''
7+
8+---
9+<!-- PLEASE FILL THESE FIELDS, IT REALLY HELPS THE MAINTAINERS OF SEARX -->
10+
11+**Is your feature request related to a problem? Please describe.**
12+<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
13+
14+**Describe the solution you'd like**
15+<!-- A clear and concise description of what you want to happen. -->
16+
17+**Describe alternatives you've considered**
18+<!-- A clear and concise description of any alternative solutions or features you've considered. -->
19+
20+**Additional context**
21+<!-- Add any other context or screenshots about the feature request here. -->
.github/dependabot.ymlnew file+10
@@ -0,0 +1,10 @@
1+# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
2+version: 2
3+updates:
4+ - package-ecosystem: "pip"
5+ directory: "/"
6+ schedule:
7+ interval: "weekly"
8+ day: "friday"
9+ open-pull-requests-limit: 5
10+ target-branch: "master"
.github/workflows/data-update.ymlnew file+66
@@ -0,0 +1,66 @@
1+name: "Update searx.data"
2+on:
3+ schedule:
4+ - cron: "05 06 1 * *"
5+ workflow_dispatch:
6+
7+jobs:
8+ updateData:
9+ name: Update data - ${{ matrix.fetch }}
10+ runs-on: ubuntu-20.04
11+ if: ${{ github.repository_owner == 'searx'}}
12+ strategy:
13+ matrix:
14+ fetch:
15+ - update_ahmia_blacklist.py
16+ - update_currencies.py
17+ - update_external_bangs.py
18+ - update_firefox_version.py
19+ - update_languages.py
20+ - update_wikidata_units.py
21+ steps:
22+ - name: Checkout
23+ uses: actions/checkout@v2
24+
25+ - name: Install Ubuntu packages
26+ run: |
27+ sudo ./utils/searx.sh install packages
28+
29+ - name: Set up Python
30+ uses: actions/setup-python@v2
31+ with:
32+ python-version: '3.10'
33+ architecture: 'x64'
34+
35+ - name: Install Python dependencies
36+ run: |
37+ make V=1 install
38+
39+ - name: Fetch data
40+ env:
41+ FETCH_SCRIPT: ./searx_extra/update/${{ matrix.fetch }}
42+ run: |
43+ V=1 ./manage pyenv.cmd python "$FETCH_SCRIPT"
44+
45+ - name: Create Pull Request
46+ id: cpr
47+ uses: peter-evans/create-pull-request@v3
48+ with:
49+ token: ${{ secrets.DATA_PR_TOKEN }}
50+ commit-message: Update searx.data - ${{ matrix.fetch }}
51+ committer: searx-bot <noreply@github.com>
52+ author: ${{ github.actor }} <${{ github.actor }}@users.noreply.github.com>
53+ signoff: false
54+ branch: update_data_${{ matrix.fetch }}
55+ delete-branch: true
56+ draft: false
57+ title: 'Update searx.data - ${{ matrix.fetch }}'
58+ body: |
59+ Update searx.data - ${{ matrix.fetch }}
60+ labels: |
61+ data
62+
63+ - name: Check outputs
64+ run: |
65+ echo "Pull Request Number - ${{ steps.cpr.outputs.pull-request-number }}"
66+ echo "Pull Request URL - ${{ steps.cpr.outputs.pull-request-url }}"
.github/workflows/integration.ymlnew file+151
@@ -0,0 +1,151 @@
1+name: Integration
2+
3+on:
4+ push:
5+ branches: ["master"]
6+ pull_request:
7+ branches: ["master"]
8+
9+jobs:
10+ python:
11+ name: Python ${{ matrix.python-version }}
12+ runs-on: ubuntu-20.04
13+ strategy:
14+ matrix:
15+ os: [ubuntu-20.04]
16+ python-version: [3.7, 3.8, 3.9, "3.10"]
17+ steps:
18+ - name: Checkout
19+ uses: actions/checkout@v2
20+ - name: Install Ubuntu packages
21+ run: |
22+ sudo ./utils/searx.sh install packages
23+ sudo apt install firefox
24+ - name: Set up Python
25+ uses: actions/setup-python@v2
26+ with:
27+ python-version: ${{ matrix.python-version }}
28+ architecture: 'x64'
29+ - name: Cache Python dependencies
30+ id: cache-python
31+ uses: actions/cache@v2
32+ with:
33+ path: ./local
34+ key: python-${{ matrix.os }}-${{ matrix.python-version }}-${{ hashFiles('requirements*.txt', 'setup.py') }}
35+ - name: Install Python dependencies
36+ if: steps.cache-python.outputs.cache-hit != 'true'
37+ run: |
38+ make V=1 install
39+ make V=1 gecko.driver
40+ - name: Run tests
41+ run: make V=1 ci.test
42+ - name: Test coverage
43+ run: make V=1 test.coverage
44+ - name: Store coverage result
45+ uses: actions/upload-artifact@v2
46+ with:
47+ name: coverage-${{ matrix.python-version }}
48+ path: coverage/
49+ retention-days: 60
50+
51+ themes:
52+ name: Themes
53+ runs-on: ubuntu-20.04
54+ steps:
55+ - name: Checkout
56+ uses: actions/checkout@v2
57+ - name: Install Ubuntu packages
58+ run: sudo ./utils/searx.sh install packages
59+ - name: Set up Python
60+ uses: actions/setup-python@v2
61+ with:
62+ python-version: '3.9'
63+ architecture: 'x64'
64+ - name: Cache Python dependencies
65+ id: cache-python
66+ uses: actions/cache@v2
67+ with:
68+ path: ./local
69+ key: python-ubuntu-20.04-3.9-${{ hashFiles('requirements*.txt', 'setup.py') }}
70+ - name: Install node dependencies
71+ run: make V=1 node.env
72+ - name: Build themes
73+ run: make V=1 themes.all
74+
75+ documentation:
76+ name: Documentation
77+ runs-on: ubuntu-20.04
78+ steps:
79+ - name: Checkout
80+ uses: actions/checkout@v2
81+ with:
82+ fetch-depth: '0'
83+ persist-credentials: false
84+ - name: Install Ubuntu packages
85+ run: sudo ./utils/searx.sh install buildhost
86+ - name: Set up Python
87+ uses: actions/setup-python@v2
88+ with:
89+ python-version: '3.10'
90+ architecture: 'x64'
91+ - name: Cache Python dependencies
92+ id: cache-python
93+ uses: actions/cache@v2
94+ with:
95+ path: ./local
96+ key: python-ubuntu-20.04-3.9-${{ hashFiles('requirements*.txt', 'setup.py') }}
97+ - name: Build documentation
98+ run: |
99+ make V=1 docs.clean docs.html
100+ - name: Deploy
101+ if: github.ref == 'refs/heads/master'
102+ uses: JamesIves/github-pages-deploy-action@3.7.1
103+ with:
104+ GITHUB_TOKEN: ${{ github.token }}
105+ BRANCH: gh-pages
106+ FOLDER: dist/docs
107+ CLEAN: true # Automatically remove deleted files from the deploy branch
108+
109+ dockers:
110+ name: Docker
111+ if: github.ref == 'refs/heads/master'
112+ needs:
113+ - python
114+ - themes
115+ - documentation
116+ env:
117+ DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
118+ runs-on: ubuntu-20.04
119+ steps:
120+ - name: Checkout
121+ if: env.DOCKERHUB_USERNAME != null
122+ uses: actions/checkout@v2
123+ with:
124+ # make sure "make docker.push" can get the git history
125+ fetch-depth: '0'
126+ - name: Set up Python
127+ uses: actions/setup-python@v2
128+ with:
129+ python-version: '3.9'
130+ architecture: 'x64'
131+ - name: Cache Python dependencies
132+ id: cache-python
133+ uses: actions/cache@v2
134+ with:
135+ path: ./local
136+ key: python-ubuntu-20.04-3.9-${{ hashFiles('requirements*.txt', 'setup.py') }}
137+ - name: Set up QEMU
138+ if: env.DOCKERHUB_USERNAME != null
139+ uses: docker/setup-qemu-action@v1
140+ - name: Set up Docker Buildx
141+ if: env.DOCKERHUB_USERNAME != null
142+ uses: docker/setup-buildx-action@v1
143+ - name: Login to DockerHub
144+ if: env.DOCKERHUB_USERNAME != null
145+ uses: docker/login-action@v1
146+ with:
147+ username: ${{ secrets.DOCKERHUB_USERNAME }}
148+ password: ${{ secrets.DOCKERHUB_TOKEN }}
149+ - name: Build and push
150+ if: env.DOCKERHUB_USERNAME != null
151+ run: make -e GIT_URL=$(git remote get-url origin) docker.push
.gitignore+2
@@ -26,3 +26,5 @@ dist/
2626local/
2727gh-pages/
2828searx.egg-info/
29+.env
30+geckodriver.log
.pylintrc+1−1
@@ -59,7 +59,7 @@ confidence=
5959# --enable=similarities". If you want to run only the classes checker, but have
6060# no Warning level messages displayed, use"--disable=all --enable=classes
6161# --disable=W"
62−disable=bad-whitespace, duplicate-code
62+disable=bad-whitespace, duplicate-code, consider-using-f-string
6363
6464# Enable the message, report, category or checker with the given id(s). You can
6565# either give multiple identifier separated by comma (,) or put this option
.travis.ymldeleted−77
@@ -1,77 +0,0 @@
1−os: linux
2−dist: bionic
3−language: python
4−cache:
5− - directories:
6− - $HOME/.cache/pip
7−addons:
8− firefox: "latest"
9−
10−install:
11− - env
12− - which python; python --version
13− - make V=1 install
14− - make V=1 gecko.driver
15− - make V=1 node.env
16− - make V=1 travis.codecov
17−script:
18− - make V=1 themes
19− - make V=1 test
20−after_success:
21− - make V=1 test.coverage
22− - codecov
23−
24−stages:
25− - test
26− - name: documentation
27− if: env(GITHUB_TOKEN) IS present
28− - name: docker
29− if: branch = master AND type != pull_request AND env(DOCKER_USERNAME) IS present
30−
31−jobs:
32− include:
33− - python: "3.5"
34− - python: "3.6"
35− - python: "3.7"
36− - python: "3.8"
37− - stage: documentation
38− python: "3.8"
39− addons: []
40− before_install:
41− - sudo ./utils/searx.sh install buildhost
42− install: true
43− script:
44− - SEARX_DEBUG=1 make travis-gh-pages
45− after_success: true
46− deploy:
47− provider: pages
48− edge: false
49− token: $GITHUB_TOKEN # Set in the settings page of your repository, as a secure variable
50− strategy: git
51− skip_cleanup: true
52− keep_history: true
53− local_dir: gh-pages
54− target_branch: gh-pages
55− on:
56− branch: master
57− - stage: docker
58− python: "3.8"
59− git:
60− depth: false
61− services:
62− - docker
63− addons: []
64− before_install: true
65− install: true
66− script:
67− - echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin
68− - make -e GIT_URL=$(git remote get-url origin) docker.push
69− after_success: true
70−
71−notifications:
72− irc:
73− channels:
74− - "irc.freenode.org#searx"
75− template:
76− - "%{repository}/#%{build_number}/%{branch} (%{author}): %{message} %{build_url}"
77− on_success: change
AUTHORS.rst+36−4
@@ -1,4 +1,4 @@
1−Searx was created by Adam Tauber and is maintained by Adam Tauber, Alexandre Flament, Noémi Ványi, @pofilo, Gaspard d'Hautefeuille and Markus Heiser.
1+Searx was created by Adam Tauber and is maintained by Adam Tauber, Noémi Ványi, @pofilo, Gaspard d'Hautefeuille and Émilien Devos.
22
33Major contributing authors:
44
@@ -11,8 +11,10 @@ Major contributing authors:
1111- Marc Abonce Seguin @a01200356
1212- @pofilo
1313- Markus Heiser @return42
14+- Émilien Devos @unixfox
15+- Alexandre Flament
1416
15−People who have submitted patches/translates, reported bugs, consulted features or
17+People who have submitted patches/translations, reported bugs, consulted features or
1618generally made searx better:
1719
1820- Laszlo Hammerl
@@ -65,7 +67,6 @@ generally made searx better:
6567- @firebovine
6668- Lorenzo J. Lucchini @luccoj
6769- @eig8phei
68−- Joachim Cherqui
6970- @maxigas
7071- Jannik Winkel @kiney
7172- @juanitobananas
@@ -110,7 +111,6 @@ generally made searx better:
110111- Nick Espig @nachtalb
111112- Rachmadani Haryono @rachmadaniHaryono
112113- Frank de Lange @yetangitu
113−- Émilien Devos @unixfox
114114- Nicolas Gelot @nfk
115115- @volth
116116- Mathieu Brunot @madmath03
@@ -134,3 +134,35 @@ generally made searx better:
134134- Sophie Tauchert @999eagle
135135- @bauruine
136136- Michael Ilsaas `<https://mikeri.net>`_
137+- @renyhp
138+- rachmadani haryono @rachmadaniHaryono
139+- Mohamad Safadieh @msafadieh
140+- @gardouille
141+- @resynth1943
142+- @Eliesemoule
143+- @gardouille
144+- @GazoilKerozen
145+- Lukáš Kucharczyk @KucharczykL
146+- Lynda Lopez @lyndalopez544
147+- M. Efe Çetin @efectn
148+- Nícholas Kegler @nicholasks
149+- @pierrechtux
150+- Scott Wallace @scottwallacesh
151+- @Singustromo
152+- @TheEvilSkeleton
153+- @Wonderfall
154+- @mrwormo
155+- Xiaoyu WEI @xywei
156+- @joshu9h
157+- Daniel Hones
158+- @cyclaero
159+- @thezeroalpha
160+- @Tobi823
161+- @archiecodes
162+- @BBaoVanC
163+- @datagram1
164+- @lucky13820
165+- @jhigginbotham
166+- @xenrox
167+- @OliveiraHermogenes
168+- Paul Alcock @Guilvareux
CHANGELOG.rst+283
@@ -1,3 +1,286 @@
1+1.0.0 2021.03.27
2+================
3+
4+Core
5+~~~~
6+
7+- drop support for Python 3.5 ( #2459 )
8+- add support for Python 3.9 ( #2397 #2459 )
9+- update Python dependencies ( #2428 #2459 #2206 ) ⚠️ pyopenssl is not longer required
10+- automatic update of searx.data ( #2555 #2585 #2595 #2592 #2600 )
11+- update searx.data ( #2604 #2605 #2606 #2607 #2415 )
12+- add ability to send engine data to subsequent requests ( #2615 )
13+- add checker ( #2419 #2476 #2481 #2682 #2682 #2657 )
14+- by default allow only HTTPS, not HTTP ( #2641 #2659 )
15+- replace /translations.js with an embedded JSON ( #2660 )
16+- activate raise_for_error by default ( #2557 )
17+- don't dump traceback of SearxEngineResponseException on init ( #2635 )
18+
19+Documentation
20+~~~~~~~~~~~~~
21+
22+- update nginx configuration ( #2618 )
23+- document workaround for using 2 languages simultaneously ( #2479 )
24+- improve admin-docs about result proxy (morty) configuration ( #2509 )
25+- fixed typo ( #2457 )
26+
27+New settings.yml
28+~~~~~~~~~~~~~~~~
29+
30+- `general.contact_url` : add link to contact instance maintainer to footer of each page ( #2391 14c7cc0e118f1d0873b32b34793cdec2c5c9c13e #2412 )
31+- `brand` : move brand options from Makefile to settings.yml ( #2408 #2473 )
32+
33+Themes
34+~~~~~~
35+
36+- oscar: Hide links panel in mobile screens ( #2458 )
37+- oscar: upgrade dependencies ( #2346 #2673 #2662 )
38+- remove legacy, courgette and pix-art themes ( #2344 )
39+- add hyperlink to searx instances list in error message ( #2387 )
40+- preferences: a tooltip is shown when the mouse is over the engine names ( #2661 )
41+- Ignore double-quotes when highlighting query parts ( #2553 )
42+- update autocomplete ( #2593 )
43+
44+New engines
45+~~~~~~~~~~~
46+
47+- ccengine ( #2533 )
48+- mediathekviewweb ( #2541 )
49+- solidtorrents ( #2626 )
50+- solr ( #2652 )
51+- rumble ( #2588 )
52+- wiby.me ( #2452 )
53+
54+Fixed engines
55+~~~~~~~~~~~~~
56+
57+- apk_mirror ( #2556 #2642 )
58+- bing ( #2602 )
59+- duckduckgo ( #2560 #2559 )
60+- library genesis ( #2448 )
61+- ina ( 0ba71c3644c4d20f70528c10eed1385399ec1c82 )
62+- invidious ( #2451 )
63+- json_engine ( #2562 )
64+- google ( #2482 )
65+- google images ( #2482 )
66+- google play apps ( 88657fe9c2a41b9be38ee5146e5870672416db12 )
67+- google play movies ( 50ba2b9e87ef61e96da124f906d3aff4c7870e3f )
68+- google news ( #2483 #2498 )
69+- google scholar ( #2611 )
70+- google video ( #2482 )
71+- hoogle ( 6255b33c9dcf0d28f0a3307af988565f69259ce2 )
72+- naver ( #2542 )
73+- semantic schollar ( f596f5767bed915a5c3bed59ae26283e53f975ca f596f5767bed915a5c3bed59ae26283e53f975ca )
74+- startpage ( #2396 )
75+- seznam ( #2564 28286cf3f2308113bf440fb6e7cf326c6ed07889 )
76+- wikipedia ( #2554 #2565 #2681 #2681 )
77+- yacy ( #2669 )
78+- yahoo news ( #2640 #2655 )
79+
80+Updated engines
81+~~~~~~~~~~~~~~~
82+
83+- duckduckgo ( 5f450fda74e80bf350eb1493f66cfa61deaf5cea )
84+- geektimes ( 45f0e1a859fa12ce2ae0c24dc356922fcad50c8d )
85+- lobste.rs ( 06b754ad67aa6066aed6df77b5ffb74aabebb040 )
86+- soundcloud ( #2671 )
87+- peertube ( #2570 )
88+- recoll ( #2539 )
89+- yggtorrent ( #2573 )
90+
91+Removed engines
92+~~~~~~~~~~~~~~~
93+
94+- acgsou ( #2654 )
95+- google_play_music ( #2558 )
96+- metager ( #2538 )
97+- voat ( #2445 )
98+- yandex ( #2566 )
99+
100+Bug fixes
101+~~~~~~~~~
102+
103+- Fix empty colon in query from selecting Chinese ( #2454 )
104+- Get correct locale with country from browser ( #2531 )
105+
106+Code refactoring / reduce the technical debt
107+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
108+
109+- refactor searx.search.SearchQuery and searx.search.EngineRef ( #2398 )
110+- dynamically set language_support variable ( #2499 )
111+- engines: add about variable ( #2460 )
112+- processors ( #2225 5c6a5407a0b124c3323e73c33b81ec1fbd7d2fce )
113+- remove Fabric file ( #2494 )
114+- use unittest from py3, remove unittest2 from py2 ( #2608 )
115+
116+Github
117+~~~~~~
118+
119+- add notice for the issue templates ( #2447 )
120+- every Sunday, call utils/fetch_*.py scripts and create a PR automatically ( #2500 728e09676400221a064627509a31470d8f6e33bf )
121+- minor change: replace "travis" by "CI" ( #2528 )
122+
123+Build scripts
124+~~~~~~~~~~~~~
125+
126+- update secret key check ( #2411 )
127+- fix makefile targets `books/{name}.*` and `books/user.pdf` ( #2420 #2530 )
128+- upload-pypi-test & linuxdoc has been released on PyPi ( #2456 )
129+- fix makefile target `gh-pages` : flatten history of branch gh.pages ( #2514 )
130+- optimize creation of the virtualenv & pyenvinstall targets ( #2421 )
131+- update pyenv pyenvinstall Make targets ( #2517 )
132+- makefile.python: remove duplicate pyenv-(un)install targets ( #2418 )
133+- [fix] make targets engines.languages and useragents.update ( #2643 )
134+- [fix] utils/serax.sh create_pyenv() - drop duplicate 'pip install .' ( #2621 )
135+
136+Install scripts
137+~~~~~~~~~~~~~~~
138+
139+- drop Ubuntu 16.04 (Xenial Xerus) support ( #2619 )
140+- replace ubu1910 image by ubu2010 image ( #2435 )
141+- LXC switch to Fedora 33 / Fedora 31 reached its EOL #2634 ( #2634 )
142+- add package which to CentOS-7 boilerplate ( #2623 )
143+- use SEARX_SETTINGS_TEMPLATE from .config environment ( #2417 )
144+- determine path to makefile.lxc in a LXC ( #2399 )
145+- remove unused code ( #2401 #2497 )
146+- support git versions <v2.22 ( #2620 )
147+
148+Announcement
149+~~~~~~~~~~~~
150+
151+We, the searx maintainer team, would like to say a huge thank you for everybody who had been involved in the development of searx or supported us in the past 7 years - making our first stable release available. Special thanks to [NLNet](https://nlnet.nl) for sponsoring multiple features of this release.
152+
153+
154+0.18.0 2020.12.14
155+=================
156+
157+Core
158+~~~~
159+
160+- drop Python 2 support ( #2137 #2244 )
161+- separate index and search routes ( #1681 ). ⚠️ add & remove your searx instance(s) from your browser.
162+- add external_bang ( #2027 #2043 #2059 )
163+- add external plugins supports ( #2074 )
164+- add plugin converting strings into hash digests ( #1246 )
165+- new category: Onions ( #565 )
166+- allow searx query parts anywhere in the query ( commit aa3c18dda9329fff875328f6ba97483c417b149a 2aef38c3b9d1fe93e9d665a49b10151d63d92392 )
167+- preferences: use base_url for prefix of sharing 'currenly saved preferences' (#1249 )
168+- upgrade to request 2.24.0, pyopenssl is optional ( #2199 )
169+- force admins to set secret_key if debug mode is disabled ( #2256 )
170+- standalone searx update ( #1591 )
171+- architecture clean up ( #2140 #2185 #2195 #2196 #2198 #2189 #2208 #2239 #2241 #2246 #2248 )
172+- record detail about engine error ( #2332 #2375 #2350 ). Add a new API endpoint: ``/stats/errors``.
173+- display if an engine does not support HTTPS requests ( #2373 )
174+
175+New settings.yml
176+~~~~~~~~~~~~~~~~
177+
178+- ``use_default_settings``: user settings can relied on the default settings ( #2291 #2362 #2349 )
179+- ``ui.results_on_new_tab: False`` - for opening result links in a new tab ( #2167 )
180+- ``ui.advanced_search`` - add preference for displaying advanced settings ( #2327 )
181+- ``server.method: "POST"`` - Make default query submission method configurable ( #2130 )
182+- ``server.default_http_headers`` - add default http headers ( #2295 )
183+- ``engines.*.proxies`` - Using proxy only for specific engines ( #1827 #2319 ), see https://searx.github.io/searx/dev/engine_overview.html#settings-yml
184+- ``enabled_plugins`` - Enabled plugins ( a05c660e3036ad8d02072fc6731af54c2ed6151c )
185+- ``preferences.lock`` - Let admins lock user preferences ( #2270 )
186+
187+Oscar theme
188+~~~~~~~~~~~
189+
190+- update infobox ( #2131 )
191+
192+ - Make infoboxes shorter by default.
193+ - Hide the main image by default as well and set a maximum height even when expanded.
194+ - Add a toggle at the bottom of the infobox to expand it or to shrink it again.
195+ - Fix pointhi style
196+- query suggestion does not keep the language tag of the original query ( #1314 )
197+- fix the clear button ( #2306 )
198+
199+Simple theme
200+~~~~~~~~~~~~
201+
202+- Fix autocomplete ( #2205 )
203+
204+New engines
205+~~~~~~~~~~~
206+
207+- ahmia, not_evil ( #565 )
208+- codeberg ( #2104 )
209+- command line engines: git grep, find, etc. ( #2128 #2250 )
210+- elasticsearch ( #2292 )
211+- metager ( #2139 )
212+- naver ( #1912 )
213+- opensemanticsearch ( #2271 )
214+- peertube ( #2109 )
215+- recoll (#2325)
216+- sepiasearch ( #2227 )
217+
218+Updated engines
219+~~~~~~~~~~~~~~~
220+
221+- digg ( #2285 )
222+- dbpedia ( #2352 )
223+- duckduckgo_definitions ( #2224 #2356 )
224+- duden ( #2359 )
225+- invidious ( #2116 )
226+- libgen ( #2360 )
227+- photon ( #2336 )
228+- soundclound ( #2365 )
229+- wikipedia ( #2178 #2363 #2354 )
230+- wikidata ( #2151 #2224 #2353 ) - faster response time
231+- yaCy ( #2255 ) - support HTTP digest authentication.
232+- youtube_noapi ( #2364 )
233+
234+Fixed engines
235+~~~~~~~~~~~~~
236+
237+- 1x ( #2361 )
238+- answer 'random sha256' ( #2121 )
239+- bing image ( #1496 )
240+- duckduckgo ( #2254 )
241+- genius ( #2371 )
242+- google ( #2236 )
243+- google image ( #2115 )
244+- lobste.rs ( #2253 )
245+- piratebay ( #2133 )
246+- startpage ( #2385 )
247+- torrentz ( #2101 )
248+
249+Removed engines
250+~~~~~~~~~~~~~~~
251+
252+- filecrop ( #2352 )
253+- searchcode_doc ( #2372 )
254+- seedpeer ( #2366 )
255+- twitter ( #2372 )
256+- yggtorrent ( #2099 #2375 )
257+
258+Install scripts & documentation
259+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
260+
261+- install script & documentation ( #2384 #2380 #2362 #2287 #2283 #2277 #2223 #2211 #2118 #2117 #2063 )
262+
263+Docker image
264+~~~~~~~~~~~~
265+
266+- use Alpine 3.12 ( #1983 )
267+- uwsgi serves the static files directly. ( #1865 )
268+- fix k8s support ( #2099 )
269+- make docker produces clean tag version ( #2182 )
270+
271+Bug fixes
272+~~~~~~~~~
273+
274+- searx.utils.HTMLTextExtractor: invalid HTML don't raise an Exception ( #2190 )
275+- Fix static URL ( commit da8b227044f45127f705f6ea94a72d368eea73bb )
276+- Fix autocomplete ( #2127 )
277+- Fix opensearch.xml ( #2132 #2247 )
278+- Fix documentation build ( #2237 )
279+- Some fixes in the fetch languages script ( #2212 )
280+
281+Special thanks to `NLNet <https://nlnet.nl>`__ for sponsoring multiple features of this release.
282+
283+
12840.17.0 2020.07.09
2285=================
3286
CONTRIBUTING.md+2−2
@@ -2,8 +2,8 @@
22
33## Resources in the documentation
44
5−* [Development quickstart](http://searx.github.io/searx/dev/contribution_guide.html)
6−* [Contribution guide](http://searx.github.io/searx/dev/contribution_guide.html)
5+* [Development quickstart](https://searx.github.io/searx/dev/contribution_guide.html)
6+* [Contribution guide](https://searx.github.io/searx/dev/contribution_guide.html)
77
88## Submitting PRs
99
Dockerfile+17−15
@@ -1,29 +1,22 @@
1−FROM alpine:3.12
1+FROM alpine:3.15
22ENTRYPOINT ["/sbin/tini","--","/usr/local/searx/dockerfiles/docker-entrypoint.sh"]
33EXPOSE 8080
44VOLUME /etc/searx
55VOLUME /var/log/uwsgi
66
7−ARG GIT_URL=unknown
8−ARG VERSION_GITCOMMIT=unknown
9−ARG SEARX_GIT_VERSION=unknown
10−
117ARG SEARX_GID=977
128ARG SEARX_UID=977
139
1410RUN addgroup -g ${SEARX_GID} searx && \
1511 adduser -u ${SEARX_UID} -D -h /usr/local/searx -s /bin/sh -G searx searx
1612
17−ARG TIMESTAMP_SETTINGS=0
18−ARG TIMESTAMP_UWSGI=0
19−ARG LABEL_VCS_REF=
20−ARG LABEL_VCS_URL=
21−
2213ENV INSTANCE_NAME=searx \
2314 AUTOCOMPLETE= \
2415 BASE_URL= \
2516 MORTY_KEY= \
26− MORTY_URL=
17+ MORTY_URL= \
18+ SEARX_SETTINGS_PATH=/etc/searx/settings.yml \
19+ UWSGI_SETTINGS_PATH=/etc/searx/uwsgi.ini
2720
2821WORKDIR /usr/local/searx
2922
@@ -53,14 +46,19 @@ RUN apk upgrade --no-cache \
5346 uwsgi \
5447 uwsgi-python3 \
5548 brotli \
56− && pip3 install --upgrade pip \
49+ && pip3 install --upgrade pip wheel setuptools \
5750 && pip3 install --no-cache -r requirements.txt \
5851 && apk del build-dependencies \
5952 && rm -rf /root/.cache
6053
61−COPY --chown=searx:searx . .
54+COPY searx ./searx
55+COPY dockerfiles ./dockerfiles
6256
63−RUN su searx -c "/usr/bin/python3 -m compileall -q searx"; \
57+ARG TIMESTAMP_SETTINGS=0
58+ARG TIMESTAMP_UWSGI=0
59+ARG VERSION_GITCOMMIT=unknown
60+
61+RUN /usr/bin/python3 -m compileall -q searx; \
6462 touch -c --date=@${TIMESTAMP_SETTINGS} searx/settings.yml; \
6563 touch -c --date=@${TIMESTAMP_UWSGI} dockerfiles/uwsgi.ini; \
6664 if [ ! -z $VERSION_GITCOMMIT ]; then\
@@ -70,8 +68,12 @@ RUN su searx -c "/usr/bin/python3 -m compileall -q searx"; \
7068 -o -name '*.svg' -o -name '*.ttf' -o -name '*.eot' \) \
7169 -type f -exec gzip -9 -k {} \+ -exec brotli --best {} \+
7270
73−# Keep this argument at the end since it change each time
71+# Keep these arguments at the end to prevent redundant layer rebuilds
7472ARG LABEL_DATE=
73+ARG GIT_URL=unknown
74+ARG SEARX_GIT_VERSION=unknown
75+ARG LABEL_VCS_REF=
76+ARG LABEL_VCS_URL=
7577LABEL maintainer="searx <${GIT_URL}>" \
7678 description="A privacy-respecting, hackable metasearch engine." \
7779 version="${SEARX_GIT_VERSION}" \
Makefile+78−239
@@ -1,268 +1,107 @@
11# -*- coding: utf-8; mode: makefile-gmake -*-
2−.DEFAULT_GOAL=help
2+# SPDX-License-Identifier: AGPL-3.0-or-later
33
4−# START Makefile setup
5−export GIT_URL=https://github.com/searx/searx
6−export GIT_BRANCH=master
7−export SEARX_URL=https://searx.me
8−export DOCS_URL=https://searx.github.io/searx
9−# END Makefile setup
4+.DEFAULT_GOAL=help
5+export MTOOLS=./manage
106
117include utils/makefile.include
128
13−PYOBJECTS = searx
14−DOC = docs
15−PY_SETUP_EXTRAS ?= \[test\]
16−
17−include utils/makefile.python
18−include utils/makefile.sphinx
19−
209all: clean install
2110
22−PHONY += help-min help-all help
23−
24−help: help-min
25− @echo ''
26− @echo 'to get more help: make help-all'
27−
28−help-min:
29− @echo ' test - run developer tests'
30− @echo ' docs - build documentation'
31− @echo ' docs-live - autobuild HTML documentation while editing'
32− @echo ' run - run developer instance'
33− @echo ' install - developer install (./local)'
34− @echo ' uninstall - uninstall (./local)'
35− @echo ' gh-pages - build docs & deploy on gh-pages branch'
36− @echo ' clean - drop builds and environments'
37− @echo ' project - re-build generic files of the searx project'
38− @echo ' buildenv - re-build environment files (aka brand)'
39− @echo ' themes - re-build build the source of the themes'
40− @echo ' docker - build Docker image'
41− @echo ' node.env - download & install npm dependencies locally'
42− @echo ''
43− @echo 'environment'
44− @echo ' SEARX_URL = $(SEARX_URL)'
45− @echo ' GIT_URL = $(GIT_URL)'
46− @echo ' DOCS_URL = $(DOCS_URL)'
47− @echo ''
48− @$(MAKE) -e -s make-help
11+PHONY += help
4912
50−help-all: help-min
51− @echo ''
52− @$(MAKE) -e -s python-help
53− @echo ''
54− @$(MAKE) -e -s docs-help
13+help:
14+ @./manage --help
15+ @echo '----'
16+ @echo 'run - run developer instance'
17+ @echo 'install - developer install of searx into virtualenv'
18+ @echo 'uninstall - uninstall developer installation'
19+ @echo 'clean - clean up working tree'
20+ @echo 'search.checker - check search engines'
21+ @echo 'test - run shell & CI tests'
22+ @echo 'test.sh - test shell scripts'
23+ @echo 'ci.test - run CI tests'
5524
56−PHONY += install
57−install: buildenv pyenvinstall
58−
59−PHONY += uninstall
60−uninstall: pyenvuninstall
61−
62−PHONY += clean
63−clean: pyclean docs-clean node.clean test.clean
64− $(call cmd,common_clean)
6525
6626PHONY += run
67−run: buildenv pyenvinstall
27+run: install
6828 $(Q) ( \
6929 sleep 2 ; \
7030 xdg-open http://127.0.0.1:8888/ ; \
7131 ) &
72− SEARX_DEBUG=1 $(PY_ENV)/bin/python ./searx/webapp.py
73−
74−# docs
75−# ----
76−
77−sphinx-doc-prebuilds:: buildenv pyenvinstall prebuild-includes
78−
79−PHONY += docs
80−docs: sphinx-doc-prebuilds
81− $(call cmd,sphinx,html,docs,docs)
82−
83−PHONY += docs-live
84−docs-live: sphinx-doc-prebuilds
85− $(call cmd,sphinx_autobuild,html,docs,docs)
86−
87−PHONY += prebuild-includes
88−prebuild-includes:
89− $(Q)mkdir -p $(DOCS_BUILD)/includes
90− $(Q)./utils/searx.sh doc | cat > $(DOCS_BUILD)/includes/searx.rst
91− $(Q)./utils/filtron.sh doc | cat > $(DOCS_BUILD)/includes/filtron.rst
92− $(Q)./utils/morty.sh doc | cat > $(DOCS_BUILD)/includes/morty.rst
93−
94−
95−$(GH_PAGES)::
96− @echo "doc available at --> $(DOCS_URL)"
97−
98−# update project files
99−# --------------------
100−
101−PHONY += project engines.languages useragents.update buildenv
102−
103−project: buildenv useragents.update engines.languages
104−
105−engines.languages: pyenvinstall
106− $(Q)echo "fetch languages .."
107− $(Q)$(PY_ENV_ACT); python utils/fetch_languages.py
108− $(Q)echo "update searx/data/engines_languages.json"
109− $(Q)mv engines_languages.json searx/data/engines_languages.json
110− $(Q)echo "update searx/languages.py"
111− $(Q)mv languages.py searx/languages.py
112−
113−useragents.update: pyenvinstall
114− $(Q)echo "Update searx/data/useragents.json with the most recent versions of Firefox."
115− $(Q)$(PY_ENV_ACT); python utils/fetch_firefox_version.py
116−
117−buildenv:
118− $(Q)echo "build searx/brand.py"
119− $(Q)echo "GIT_URL = '$(GIT_URL)'" > searx/brand.py
120− $(Q)echo "GIT_BRANCH = '$(GIT_BRANCH)'" >> searx/brand.py
121− $(Q)echo "ISSUE_URL = 'https://github.com/searx/searx/issues'" >> searx/brand.py
122− $(Q)echo "SEARX_URL = '$(SEARX_URL)'" >> searx/brand.py
123− $(Q)echo "DOCS_URL = '$(DOCS_URL)'" >> searx/brand.py
124− $(Q)echo "PUBLIC_INSTANCES = 'https://searx.space'" >> searx/brand.py
125− $(Q)echo "build utils/brand.env"
126− $(Q)echo "export GIT_URL='$(GIT_URL)'" > utils/brand.env
127− $(Q)echo "export GIT_BRANCH='$(GIT_BRANCH)'" >> utils/brand.env
128− $(Q)echo "export ISSUE_URL='https://github.com/searx/searx/issues'" >> utils/brand.env
129− $(Q)echo "export SEARX_URL='$(SEARX_URL)'" >> utils/brand.env
130− $(Q)echo "export DOCS_URL='$(DOCS_URL)'" >> utils/brand.env
131− $(Q)echo "export PUBLIC_INSTANCES='https://searx.space'" >> utils/brand.env
132−
32+ SEARX_DEBUG=1 ./manage pyenv.cmd python ./searx/webapp.py
13333
134−# node / npm
135−# ----------
34+PHONY += install uninstall
35+install uninstall:
36+ $(Q)./manage pyenv.$@
13637
137−node.env: buildenv
138− $(Q)./manage.sh npm_packages
139−
140−node.clean:
141− $(Q)echo "CLEAN locally installed npm dependencies"
142− $(Q)rm -rf \
143− ./node_modules \
144− ./package-lock.json \
145− ./searx/static/themes/oscar/package-lock.json \
146− ./searx/static/themes/oscar/node_modules \
147− ./searx/static/themes/simple/package-lock.json \
148− ./searx/static/themes/simple/node_modules
149−
150−# build themes
151−# ------------
152−
153−PHONY += themes.bootstrap themes themes.oscar themes.simple themes.legacy themes.courgette themes.pixart
154−themes: buildenv themes.bootstrap themes.oscar themes.simple themes.legacy themes.courgette themes.pixart
155−
156−quiet_cmd_lessc = LESSC $3
157− cmd_lessc = PATH="$$(npm bin):$$PATH" \
158− lessc --clean-css="--s1 --advanced --compatibility=ie9" "searx/static/$2" "searx/static/$3"
159−
160−quiet_cmd_grunt = GRUNT $2
161− cmd_grunt = PATH="$$(npm bin):$$PATH" \
162− grunt --gruntfile "$2"
163−
164−themes.oscar: node.env
165− $(Q)echo '[!] build oscar theme'
166− $(call cmd,grunt,searx/static/themes/oscar/gruntfile.js)
167−
168−themes.simple: node.env
169− $(Q)echo '[!] build simple theme'
170− $(call cmd,grunt,searx/static/themes/simple/gruntfile.js)
171−
172−themes.legacy: node.env
173− $(Q)echo '[!] build legacy theme'
174− $(call cmd,lessc,themes/legacy/less/style-rtl.less,themes/legacy/css/style-rtl.css)
175− $(call cmd,lessc,themes/legacy/less/style.less,themes/legacy/css/style.css)
176−
177−themes.courgette: node.env
178− $(Q)echo '[!] build courgette theme'
179− $(call cmd,lessc,themes/courgette/less/style.less,themes/courgette/css/style.css)
180− $(call cmd,lessc,themes/courgette/less/style-rtl.less,themes/courgette/css/style-rtl.css)
181−
182−themes.pixart: node.env
183− $(Q)echo '[!] build pixart theme'
184− $(call cmd,lessc,themes/pix-art/less/style.less,themes/pix-art/css/style.css)
185−
186−themes.bootstrap: node.env
187− $(call cmd,lessc,less/bootstrap/bootstrap.less,css/bootstrap.min.css)
188−
189−
190−# docker
191−# ------
192−
193−PHONY += docker
194−docker: buildenv
195− $(Q)./manage.sh docker_build
196−
197−docker.push: buildenv
198− $(Q)./manage.sh docker_build push
199−
200−# gecko
201−# -----
202−
203−PHONY += gecko.driver
204−gecko.driver:
205− $(PY_ENV_ACT); ./manage.sh install_geckodriver
206−
207−# test
208−# ----
209−
210−PHONY += test test.sh test.pylint test.pep8 test.unit test.coverage test.robot
211−test: buildenv test.pylint test.pep8 test.unit gecko.driver test.robot
212−
213−# TODO: balance linting with pylint
38+PHONY += clean
39+clean: py.clean docs.clean node.clean test.clean
40+ $(Q)./manage build_msg CLEAN "common files"
41+ $(Q)find . -name '*.orig' -exec rm -f {} +
42+ $(Q)find . -name '*.rej' -exec rm -f {} +
43+ $(Q)find . -name '*~' -exec rm -f {} +
44+ $(Q)find . -name '*.bak' -exec rm -f {} +
45+
46+PHONY += search.checker search.checker.%
47+search.checker: install
48+ $(Q)./manage pyenv.cmd searx-checker -v
49+
50+search.checker.%: install
51+ $(Q)./manage pyenv.cmd searx-checker -v "$(subst _, ,$(patsubst search.checker.%,%,$@))"
52+
53+PHONY += ci.test test test.sh
54+ci.test: test.pep8 test.pylint test.unit test.robot
55+test: ci.test
56+test.sh:
57+ $(Q)shellcheck -x -s bash \
58+ utils/brand.env \
59+ ./manage \
60+ utils/lib.sh \
61+ utils/filtron.sh \
62+ utils/searx.sh \
63+ utils/morty.sh \
64+ utils/lxc.sh \
65+ utils/lxc-searx.env \
66+ .config.sh
67+ $(Q)./manage build_msg TEST "$@ OK"
21468
215−test.pylint: pyenvinstall
216− $(call cmd,pylint,\
217− searx/preferences.py \
218− searx/testing.py \
219− searx/engines/gigablast.py \
220− )
22169
222−# ignored rules:
223−# E402 module level import not at top of file
224−# W503 line break before binary operator
70+# wrap ./manage script
22571
226−# ubu1604: uses shellcheck v0.3.7 (from 04/2015), no longer supported!
227−test.sh:
228− shellcheck -x -s bash utils/brand.env
229− shellcheck -x utils/lib.sh
230− shellcheck -x utils/filtron.sh
231− shellcheck -x utils/searx.sh
232− shellcheck -x utils/morty.sh
233− shellcheck -x utils/lxc.sh
234− shellcheck -x utils/lxc-searx.env
235− shellcheck -x .config.sh
72+MANAGE += buildenv
73+MANAGE += babel.compile
74+MANAGE += data.all data.languages data.useragents
75+MANAGE += docs.html docs.live docs.gh-pages docs.prebuild docs.clean
76+MANAGE += docker.build docker.push
77+MANAGE += gecko.driver
78+MANAGE += node.env node.clean
79+MANAGE += py.build py.clean
80+MANAGE += pyenv pyenv.install pyenv.uninstall
81+MANAGE += pypi.upload pypi.upload.test
82+MANAGE += test.pylint test.pep8 test.unit test.coverage test.robot test.clean
83+MANAGE += themes.all themes.oscar themes.simple themes.bootstrap
23684
237−test.pep8: pyenvinstall
238− @echo "TEST pep8"
239− $(Q)$(PY_ENV_ACT); pep8 --exclude='searx/static, searx/languages.py, searx/engines/gigablast.py' \
240− --max-line-length=120 --ignore "E402,W503" searx tests
85+PHONY += $(MANAGE)
24186
242−test.unit: pyenvinstall
243− @echo "TEST tests/unit"
244− $(Q)$(PY_ENV_ACT); python -m nose2 -s tests/unit
87+$(MANAGE):
88+ $(Q)$(MTOOLS) $@
24589
246−test.coverage: pyenvinstall
247− @echo "TEST unit test coverage"
248− $(Q)$(PY_ENV_ACT); \
249− python -m nose2 -C --log-capture --with-coverage --coverage searx -s tests/unit \
250− && coverage report \
251− && coverage html \
90+# deprecated
25291
253−test.robot: pyenvinstall gecko.driver
254− @echo "TEST robot"
255− $(Q)$(PY_ENV_ACT); PYTHONPATH=. python searx/testing.py robot
92+PHONY += docs docs-clean docs-live docker themes
25693
257−test.clean:
258− @echo "CLEAN intermediate test stuff"
259− $(Q)rm -rf geckodriver.log .coverage coverage/
94+docs: docs.html
95+ $(Q)./manage build_msg WARN $@ is deprecated use docs.html
26096
97+docs-clean: docs.clean
98+ $(Q)./manage build_msg WARN $@ is deprecated use docs.clean
26199
262−# travis
263−# ------
100+docs-live: docs.live
101+ $(Q)./manage build_msg WARN $@ is deprecated use docs.live
264102
265−travis.codecov:
266− $(Q)$(PY_ENV_BIN)/python -m pip install codecov
103+docker: docker.build
104+ $(Q)./manage build_msg WARN $@ is deprecated use docker.build
267105
268−.PHONY: $(PHONY)
106+themes: themes.all
107+ $(Q)./manage build_msg WARN $@ is deprecated use themes.all
README.rst+2−2
@@ -17,7 +17,7 @@
1717|OpenCollective searx backers|
1818|OpenCollective searx sponsors|
1919
20−Privacy-respecting, hackable `metasearch engine`_ / *pronunciation* **səːks**.
20+Privacy-respecting, hackable `metasearch engine`_ / *pronunciation* **sɜːks**.
2121
2222.. _metasearch engine: https://en.wikipedia.org/wiki/Metasearch_engine
2323
@@ -61,7 +61,7 @@ our homepage_.
6161.. _homepage: https://searx.github.io/searx
6262
6363contact:
64− openhub_ // twitter_ // IRC: #searx @ freenode
64+ openhub_ // twitter_ // IRC: #searx @ Libera (irc.libera.chat)
6565
6666.. _openhub: https://www.openhub.net/p/searx
6767.. _twitter: https://twitter.com/Searx_engine
dockerfiles/docker-entrypoint.sh+1−4
@@ -24,9 +24,6 @@ if [ -z "${BIND_ADDRESS}" ]; then
2424 export BIND_ADDRESS="${DEFAULT_BIND_ADDRESS}"
2525fi
2626
27−export UWSGI_SETTINGS_PATH=/etc/searx/uwsgi.ini
28−export SEARX_SETTINGS_PATH=/etc/searx/settings.yml
29−
3027# Parse special command line
3128# see docs/admin/installation-docker.rst
3229# display the help message without the version
@@ -66,7 +63,7 @@ patch_searx_settings() {
6663 CONF="$1"
6764
6865 # Make sure that there is trailing slash at the end of BASE_URL
69− # see http://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Shell-Parameter-Expansion
66+ # see https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Shell-Parameter-Expansion
7067 export BASE_URL="${BASE_URL%/}/"
7168
7269 # update settings.yml
dockerfiles/uwsgi.ini+3
@@ -42,3 +42,6 @@ static-map = /static=/usr/local/searx/searx/static
4242static-expires = /* 864000
4343static-gzip-all = True
4444offload-threads = %k
45+
46+# Cache
47+cache2 = name=searxcache,items=2000,blocks=2000,blocksize=4096,bitmap=1
docs/_themes/searx/static/searx.css−29
@@ -138,32 +138,3 @@ caption {
138138 caption-side: top;
139139 text-align: left;
140140}
141−
142−/* bugs since sphinx 3.1
143−
144−See sphinx-doc project, PR 7838 & 7484 with elementary patch to the basic CSS:
145−
146−- https://github.com/sphinx-doc/sphinx/issues/7838#issuecomment-646009605
147−- https://github.com/sphinx-doc/sphinx/pull/7484#issuecomment-646058972
148−
149−*/
150−
151−li > p:first-child {
152− margin-top: 0;
153−}
154−
155−li > p:last-child {
156− margin-bottom: 0;
157−}
158−
159−div.admonition dl {
160− margin-bottom: 0;
161−}
162−
163−div.sidebar {
164− clear: none;
165−}
166−
167−div.admonition, div.topic, pre {
168− clear: none;
169−}
docs/admin/buildhosts.rst+5−5
@@ -49,9 +49,9 @@ Build docs
4949 - dvisvgm_
5050
5151Most of the sphinx requirements are installed from :origin:`setup.py` and the
52−docs can be build from scratch with ``make docs``. For better math and image
53−processing additional packages are needed. The XeTeX_ needed not only for PDF
54−creation, its also needed for :ref:`math` when HTML output is build.
52+docs can be build from scratch with ``make docs.html``. For better math and
53+image processing additional packages are needed. The XeTeX_ needed not only for
54+PDF creation, its also needed for :ref:`math` when HTML output is build.
5555
5656To be able to do :ref:`sphinx:math-support` without CDNs, the math are rendered
5757as images (``sphinx.ext.imgmath`` extension).
@@ -64,10 +64,10 @@ to ``imgmath``:
6464 :start-after: # sphinx.ext.imgmath setup
6565 :end-before: # sphinx.ext.imgmath setup END
6666
67−If your docs build (``make docs``) shows warnings like this::
67+If your docs build (``make docs.html``) shows warnings like this::
6868
6969 WARNING: dot(1) not found, for better output quality install \
70− graphviz from http://www.graphviz.org
70+ graphviz from https://www.graphviz.org
7171 ..
7272 WARNING: LaTeX command 'latex' cannot be run (needed for math \
7373 display), check the imgmath_latex setting
docs/admin/engines.rst+26−13
@@ -1,14 +1,28 @@
1−.. _engines generic:
2−
31=======
42Engines
53=======
64
5+Special Engine Settings
6+=======================
7+
78.. sidebar:: Further reading ..
89
910 - :ref:`settings engine`
10− - :ref:`engine settings`
11− - :ref:`engine file`
11+ - :ref:`engine settings` & :ref:`engine file`
12+
13+.. toctree::
14+ :maxdepth: 1
15+
16+ engines/recoll.rst
17+
18+
19+.. _engines generic:
20+
21+General Engine Settings
22+=======================
23+
24+Explanation of the :ref:`general engine configuration` shown in the table
25+:ref:`configured engines`.
1226
1327============= =========== ==================== ============
1428:ref:`engine settings` :ref:`engine file`
@@ -19,7 +33,7 @@ Engine .. Paging support **P**
1933------------------------- -------------------- ------------
2034Shortcut **S** Language support **L**
2135Timeout **TO** Time range support **TR**
22−Disabled **D** Offline **O**
36+Disabled **D** Engine type **ET**
2337------------- ----------- -------------------- ------------
2438Safe search **SS**
2539------------- ----------- ---------------------------------
@@ -30,11 +44,9 @@ Disabled **D**
3044Show errors **DE**
3145============= =========== =================================
3246
33−Configuration defaults (at built time):
34−
3547.. _configured engines:
3648
37−.. jinja:: webapp
49+.. jinja:: searx
3850
3951 .. flat-table:: Engines configured at built time (defaults)
4052 :header-rows: 1
@@ -50,10 +62,10 @@ Configuration defaults (at built time):
5062 - SS
5163 - D
5264 - TR
53− - O
54− - W
55− - D
56− - DE
65+ - ET
66+ - W
67+ - D
68+ - DE
5769
5870 {% for name, mod in engines.items() %}
5971
@@ -67,9 +79,10 @@ Configuration defaults (at built time):
6779 - {{(mod.safesearch and "y") or ""}}
6880 - {{(mod.disabled and "y") or ""}}
6981 - {{(mod.time_range_support and "y") or ""}}
70− - {{(mod.offline and "y") or ""}}
82+ - {{mod.engine_type or ""}}
7183 - {{mod.weight or 1 }}
7284 - {{(mod.disabled and "y") or ""}}
7385 - {{(mod.display_error_messages and "y") or ""}}
7486
7587 {% endfor %}
88+
docs/admin/engines/recoll.rstnew file+50
@@ -0,0 +1,50 @@
1+.. _engine recoll:
2+
3+======
4+Recoll
5+======
6+
7+.. sidebar:: info
8+
9+ - `Recoll <https://www.lesbonscomptes.com/recoll/>`_
10+ - `recoll-webui <https://framagit.org/medoc92/recollwebui.git>`_
11+
12+Recoll_ is a desktop full-text search tool based on Xapian. By itself Recoll_
13+does not offer web or API access, this can be achieved using recoll-webui_
14+
15+
16+
17+Configuration
18+=============
19+
20+You must configure the following settings:
21+
22+``base_url``:
23+ Location where recoll-webui can be reached.
24+
25+``mount_prefix``:
26+ Location where the file hierarchy is mounted on your *local* filesystem.
27+
28+``dl_prefix``:
29+ Location where the file hierarchy as indexed by recoll can be reached.
30+
31+``search_dir``:
32+ Part of the indexed file hierarchy to be search, if empty the full domain is
33+ searched.
34+
35+
36+Example
37+=======
38+
39+Scenario:
40+
41+#. Recoll indexes a local filesystem mounted in ``/export/documents/reference``,
42+#. the Recoll search inteface can be reached at https://recoll.example.org/ and
43+#. the contents of this filesystem can be reached though https://download.example.org/reference
44+
45+.. code:: yaml
46+
47+ base_url: https://recoll.example.org/
48+ mount_prefix: /export/documents
49+ dl_prefix: https://download.example.org
50+ search_dir: ''
docs/admin/filtron.rst+1−1
@@ -173,7 +173,7 @@ Use it along with ``nginx`` with the following example configuration.
173173 location /searx {
174174 proxy_pass http://127.0.0.1:4004/;
175175
176− proxy_set_header Host $http_host;
176+ proxy_set_header Host $host;
177177 proxy_set_header Connection $http_connection;
178178 proxy_set_header X-Real-IP $remote_addr;
179179 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
docs/admin/installation-apache.rst+8−7
@@ -180,10 +180,6 @@ modules and create a `Location`_ configuration for the searx site. In most
180180distributions you have to un-comment the lines in the main configuration file,
181181except in :ref:`The Debian Layout`.
182182
183−To pass the HTTP HOST header
184−With ProxyPreserveHost_ the incoming Host HTTP request header is passed to the
185−proxied host.
186−
187183.. tabs::
188184
189185 .. group-tab:: Ubuntu / debian
@@ -231,6 +227,11 @@ proxied host.
231227 LoadModule proxy_module modules/mod_proxy.so
232228 LoadModule proxy_http_module modules/mod_proxy_http.so
233229
230+With ProxyPreserveHost_ the incoming Host HTTP request header is passed to the
231+proxied host.
232+
233+.. _apache searx via filtron plus morty:
234+
234235.. tabs::
235236
236237 .. group-tab:: searx via filtron plus morty
@@ -285,15 +286,15 @@ proxied host.
285286
286287 </Location>
287288
288− Note that reverse proxy advised to be used in case of single-user or
289− low-traffic instances. For a fully result proxification add :ref:`morty's
290− <searx morty>` **public URL** to your :origin:`searx/settings.yml`:
289+ For a fully result proxification add :ref:`morty's <searx morty>` **public
290+ URL** to your :origin:`searx/settings.yml`:
291291
292292 .. code:: yaml
293293
294294 result_proxy:
295295 # replace example.org with your server's public name
296296 url : https://example.org/morty
297+ key : !!binary "insert_your_morty_proxy_key_here"
297298
298299 server:
299300 image_proxy : True
docs/admin/installation-docker.rst+1−1
@@ -51,7 +51,7 @@ It's also possible to build searx from the embedded Dockerfile.
5151
5252 git clone https://github.com/searx/searx.git
5353 cd searx
54− make docker
54+ make docker.build
5555
5656
5757Public instance
docs/admin/installation-nginx.rst+15−13
@@ -9,7 +9,7 @@ Install with nginx
99.. _nginx server configuration:
1010 https://docs.nginx.com/nginx/admin-guide/web-server/web-server/#setting-up-virtual-servers
1111.. _nginx beginners guide:
12− http://nginx.org/en/docs/beginners_guide.html
12+ https://nginx.org/en/docs/beginners_guide.html
1313.. _Getting Started wiki:
1414 https://www.nginx.com/resources/wiki/start/
1515.. _uWSGI support from nginx:
@@ -163,6 +163,8 @@ Started wiki`_ is always a good resource *to keep in the pocket*.
163163 Create configuration at ``/etc/nginx/conf.d/searx`` and place a
164164 symlink to sites-enabled:
165165
166+.. _nginx searx via filtron plus morty:
167+
166168.. tabs::
167169
168170 .. group-tab:: searx via filtron plus morty
@@ -180,7 +182,7 @@ Started wiki`_ is always a good resource *to keep in the pocket*.
180182 location /searx {
181183 proxy_pass http://127.0.0.1:4004/;
182184
183− proxy_set_header Host $http_host;
185+ proxy_set_header Host $host;
184186 proxy_set_header Connection $http_connection;
185187 proxy_set_header X-Real-IP $remote_addr;
186188 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
@@ -188,8 +190,8 @@ Started wiki`_ is always a good resource *to keep in the pocket*.
188190 proxy_set_header X-Script-Name /searx;
189191 }
190192
191− location /searx/static {
192− alias /usr/local/searx/searx-src/searx/static;
193+ location /searx/static/ {
194+ alias /usr/local/searx/searx-src/searx/static/;
193195 }
194196
195197
@@ -203,28 +205,28 @@ Started wiki`_ is always a good resource *to keep in the pocket*.
203205 location /morty {
204206 proxy_pass http://127.0.0.1:3000/;
205207
206− proxy_set_header Host $http_host;
208+ proxy_set_header Host $host;
207209 proxy_set_header Connection $http_connection;
208210 proxy_set_header X-Real-IP $remote_addr;
209211 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
210212 proxy_set_header X-Scheme $scheme;
211213 }
212214
213− Note that reverse proxy advised to be used in case of single-user or
214− low-traffic instances. For a fully result proxification add :ref:`morty's
215− <searx morty>` **public URL** to your :origin:`searx/settings.yml`:
215+ For a fully result proxification add :ref:`morty's <searx morty>` **public
216+ URL** to your :origin:`searx/settings.yml`:
216217
217218 .. code:: yaml
218219
219220 result_proxy:
220221 # replace example.org with your server's public name
221222 url : https://example.org/morty
223+ key : !!binary "insert_your_morty_proxy_key_here"
222224
223225 server:
224226 image_proxy : True
225227
226228
227− .. group-tab:: proxy or uWSGI
229+ .. group-tab:: proxy or uWSGI
228230
229231 Be warned, with this setup, your instance isn't :ref:`protected <searx
230232 filtron>`. Nevertheless it is good enough for intranet usage and it is a
@@ -307,8 +309,8 @@ Started wiki`_ is always a good resource *to keep in the pocket*.
307309 proxy_buffering off;
308310 }
309311
310− location /searx/static {
311− alias /usr/local/searx/searx-src/searx/static;
312+ location /searx/static/ {
313+ alias /usr/local/searx/searx-src/searx/static/;
312314 }
313315
314316 The ``X-Script-Name /searx`` is needed by the searx implementation to
@@ -326,8 +328,8 @@ Started wiki`_ is always a good resource *to keep in the pocket*.
326328 uwsgi_pass unix:/run/uwsgi/app/searx/socket;
327329 }
328330
329− location /searx/static {
330− alias /usr/local/searx/searx-src/searx;
331+ location /searx/static/ {
332+ alias /usr/local/searx/searx-src/searx/;
331333 }
332334
333335 For searx to work correctly the ``base_url`` must be set in the
docs/admin/installation-searx.rst+34−6
@@ -52,7 +52,7 @@ In the same shell create *virtualenv*:
5252 :end-before: END create virtualenv
5353
5454To install searx's dependencies, exit the searx *bash* session you opened above
55−and restart a new. Before install, first check if your *virualenv* was sourced
55+and restart a new. Before install, first check if your *virtualenv* was sourced
5656from the login (*~/.profile*):
5757
5858.. kernel-include:: $DOCS_BUILD/includes/searx.rst
@@ -61,20 +61,48 @@ from the login (*~/.profile*):
6161
6262.. tip::
6363
64− Open a second terminal for the configuration tasks and left the ``(searx)$``
64+ Open a second terminal for the configuration tasks and leave the ``(searx)$``
6565 terminal open for the tasks below.
6666
67+
68+.. _use_default_settings.yml:
69+
6770Configuration
68−==============
71+=============
72+
73+.. sidebar:: ``use_default_settings: True``
6974
70−Create a copy of the :origin:`searx/settings.yml` configuration file in system's
71−*/etc* folder. Configure like shown below -- replace ``searx@\$(uname -n)`` with
72−a name of your choice -- *and/or* edit ``/etc/searx/settings.yml`` if necessary.
75+ - :ref:`settings global`
76+ - :ref:`settings location`
77+ - :ref:`settings use_default_settings`
78+ - :origin:`/etc/searx/settings.yml <utils/templates/etc/searx/use_default_settings.yml>`
79+
80+To create a initial ``/etc/searx/settings.yml`` you can start with a copy of the
81+file :origin:`utils/templates/etc/searx/use_default_settings.yml`. This setup
82+:ref:`use default settings <settings use_default_settings>` from
83+:origin:`searx/settings.yml` and is recommended since :pull:`2291` is merged.
84+
85+For a *minimal setup*, configure like shown below – replace ``searx@$(uname
86+-n)`` with a name of your choice, set ``ultrasecretkey`` -- *and/or* edit
87+``/etc/searx/settings.yml`` to your needs.
7388
7489.. kernel-include:: $DOCS_BUILD/includes/searx.rst
7590 :start-after: START searx config
7691 :end-before: END searx config
7792
93+.. tabs::
94+
95+ .. group-tab:: Use default settings
96+
97+ .. literalinclude:: ../../utils/templates/etc/searx/use_default_settings.yml
98+ :language: yaml
99+
100+ .. group-tab:: searx/settings.yml
101+
102+ .. literalinclude:: ../../searx/settings.yml
103+ :language: yaml
104+
105+
78106Check
79107=====
80108
docs/admin/installation.rst+20
@@ -33,12 +33,24 @@ The following will install a setup as shown in :ref:`architecture`. First you
3333need to get a clone. The clone is only needed for the installation procedure
3434and some maintenance tasks (alternatively you can create your own fork).
3535
36+For the installation procedure, use a *sudoer* login to run the scripts. If you
37+install from ``root``, take into account that the scripts are creating a
38+``searx``, a ``filtron`` and a ``morty`` user. In the installation procedure
39+these new created users do need read access to the clone of searx, which is not
40+the case if you clone into a folder below ``/root``.
41+
3642.. code:: bash
3743
3844 $ cd ~/Downloads
3945 $ git clone https://github.com/searx/searx searx
4046 $ cd searx
4147
48+.. sidebar:: further read
49+
50+ - :ref:`toolboxing`
51+ - :ref:`update searx`
52+ - :ref:`inspect searx`
53+
4254**Install** :ref:`searx service <searx.sh>`
4355
4456This installs searx as described in :ref:`installation basic`.
@@ -64,3 +76,11 @@ If all services are running fine, you can add it to your HTTP server:
6476- :ref:`installation apache`
6577- :ref:`installation nginx`
6678
79+.. _git stash: https://git-scm.com/docs/git-stash
80+
81+.. tip::
82+
83+ About script's installation options have a look at chapter :ref:`toolboxing
84+ setup`. How to brand your instance see chapter :ref:`settings global`. To
85+ *stash* your instance's setup, `git stash`_ your clone's :origin:`Makefile`
86+ and :origin:`.config.sh` file .
docs/admin/morty.rst+11−4
@@ -16,15 +16,22 @@ By default searx can only act as an image proxy for result images, but it is
1616possible to proxify all the result URLs with an external service, morty_.
1717
1818To use this feature, morty has to be installed and activated in searx's
19−``settings.yml``.
20−
21−Add the following snippet to your ``settings.yml`` and restart searx:
19+``settings.yml``. Add the following snippet to your ``settings.yml`` and
20+restart searx:
2221
2322.. code:: yaml
2423
2524 result_proxy:
2625 url : http://127.0.0.1:3000/
27− key : your_morty_proxy_key
26+ key : !!binary "insert_your_morty_proxy_key_here"
27+
28+Note that the example above (``http://127.0.0.1:3000``) is only for single-user
29+instances without a HTTP proxy. If your morty service is public, the url is the
30+address of the reverse proxy (e.g ``https://example.org/morty``).
31+
32+For more information about *result proxy* have a look at *"searx via filtron
33+plus morty"* in the :ref:`nginx <nginx searx via filtron plus morty>` and
34+:ref:`apache <apache searx via filtron plus morty>` sections.
2835
2936``url``
3037 Is the address of the running morty service.
docs/admin/plugins.rst+1−1
@@ -14,7 +14,7 @@ Configuration defaults (at built time):
1414
1515.. _configured plugins:
1616
17−.. jinja:: webapp
17+.. jinja:: searx
1818
1919 .. flat-table:: Plugins configured at built time (defaults)
2020 :header-rows: 1
docs/admin/settings.rst+268−64
@@ -9,6 +9,7 @@ file.
99
1010.. sidebar:: Further reading ..
1111
12+ - :ref:`use_default_settings.yml`
1213 - :ref:`search API`
1314
1415.. contents:: Contents
@@ -16,85 +17,93 @@ file.
1617 :local:
1718 :backlinks: entry
1819
20+.. _settings location:
21+
22+settings.yml location
23+=====================
24+
25+First, searx will try to load settings.yml from these locations:
26+
27+1. the full path specified in the ``SEARX_SETTINGS_PATH`` environment variable.
28+2. ``/etc/searx/settings.yml``
29+
30+If these files don't exist (or are empty or can't be read), searx uses the
31+:origin:`searx/settings.yml` file.
32+
33+
1934.. _settings global:
2035
2136Global Settings
2237===============
2338
39+``general:``
40+------------
41+
2442.. code:: yaml
2543
26− server:
27− port : 8888
28− secret_key : "ultrasecretkey" # change this!
29− debug : False # debug mode, only for development
30− request_timeout : 2.0 # seconds
31− base_url : False # set custom base_url (or False)
32− themes_path : "" # custom ui themes path
33− default_theme : oscar # ui theme
34− useragent_suffix : "" # suffix of searx_useragent, could contain
35− # informations like admins email address
36− image_proxy : False # proxying image results through searx
37− default_locale : "" # default interface locale
44+ general:
45+ debug : False # Debug mode, only for development
46+ instance_name : "searx" # displayed name
47+ git_url: https://github.com/searx/searx
48+ git_branch: master
49+ issue_url: https://github.com/searx/searx/issues
50+ docs_url: https://searx.github.io/searx
51+ public_instances: https://searx.space
52+ contact_url: False # mailto:contact@example.com
53+ wiki_url: https://github.com/searx/searx/wiki
54+ twitter_url: https://twitter.com/Searx_engine
3855
39− # uncomment below section if you want to use a proxy
56+``debug`` :
57+ Allow a more detailed log if you run searx directly. Display *detailed* error
58+ messages in the browser too, so this must be deactivated in production.
4059
41− #outgoing_proxies :
42− # http : http://127.0.0.1:8080
43− # https: http://127.0.0.1:8080
60+``contact_url``:
61+ Contact ``mailto:`` address or WEB form.
4462
45− # uncomment below section only if you have more than one network interface
46− # which can be the source of outgoing search requests
63+``git_url`` and ``git_branch``:
64+ Changes this, to point to your searx fork (branch).
4765
48− #source_ips:
49− # - 1.1.1.1
50− # - 1.1.1.2
66+``docs_url``
67+ If you host your own documentation, change this URL.
5168
52− locales:
53− en : English
54− de : Deutsch
55− he : Hebrew
56− hu : Magyar
57− fr : Français
58− es : Español
59− it : Italiano
60− nl : Nederlands
61− ja : 日本語 (Japanese)
62− tr : Türkçe
63− ru : Russian
64− ro : Romanian
69+``wiki_url``:
70+ Link to your wiki (or ``False``)
6571
72+``twitter_url``:
73+ Link to your tweets (or ``False``)
6674
67−``port`` :
68− Port number of the searx web application if you run it directly using ``python
69− searx/webapp.py``. Doesn't apply to searx running on Apache or Nginx.
7075
71−``secret_key`` :
72− Used for cryptography purpose.
76+``server:``
77+-----------
7378
74−``debug`` :
75− Allow a more detailed log if you run searx directly. Display *detailed* error
76− messages in the browser too, so this must be deactivated in production.
79+.. code:: yaml
7780
78−``request_timeout`` :
79− Global timeout of the requests made to others engines in seconds. A bigger
80− timeout will allow to wait for answers from slow engines, but in consequence
81− will slow searx reactivity (the result page may take the time specified in the
82− timeout to load)
81+ server:
82+ port : 8888
83+ bind_address : "127.0.0.1" # address to listen on
84+ secret_key : "ultrasecretkey" # change this!
85+ base_url : False # set custom base_url (or False)
86+ image_proxy : False # proxying image results through searx
87+ default_locale : "" # default interface locale
88+ default_theme : oscar # ui theme
89+ default_http_headers:
90+ X-Content-Type-Options : nosniff
91+ X-XSS-Protection : 1; mode=block
92+ X-Download-Options : noopen
93+ X-Robots-Tag : noindex, nofollow
94+ Referrer-Policy : no-referrer
95+
96+``port`` & ``bind_address``:
97+ Port number and *bind address* of the searx web application if you run it
98+ directly using ``python searx/webapp.py``. Doesn't apply to searx running on
99+ Apache or Nginx.
100+
101+``secret_key`` :
102+ Used for cryptography purpose.
83103
84104``base_url`` :
85105 The base URL where searx is deployed. Used to create correct inbound links.
86106
87−``themes_path`` :
88− Path to where the themes are located. If you didn't develop anything, leave it
89− blank.
90−
91−``default_theme`` :
92− Name of the theme you want to use by default on your searx instance.
93−
94−``useragent_suffix`` :
95− Suffix to the user-agent searx uses to send requests to others engines. If an
96− engine wish to block you, a contact info here may be useful to avoid that.
97−
98107``image_proxy`` :
99108 Allow your instance of searx of being able to proxy images. Uses memory space.
100109
@@ -104,16 +113,109 @@ Global Settings
104113 specific instance of searx, a locale can be defined using an ISO language
105114 code, like ``fr``, ``en``, ``de``.
106115
107−.. _requests proxies: http://requests.readthedocs.io/en/latest/user/advanced/#proxies
108−.. _PR SOCKS support: https://github.com/kennethreitz/requests/pull/478
116+``default_theme`` :
117+ Name of the theme you want to use by default on your searx instance.
118+
119+.. _HTTP headers: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
120+
121+``default_http_headers``:
122+ Set additional HTTP headers, see `#755 <https://github.com/searx/searx/issues/715>`__
123+
124+``outgoing:``
125+-------------
126+
127+.. code:: yaml
128+
129+ outgoing: # communication with search engines
130+ request_timeout : 2.0 # default timeout in seconds, can be override by engine
131+ # max_request_timeout: 10.0 # the maximum timeout in seconds
132+ useragent_suffix : "" # informations like an email address to the administrator
133+ pool_connections : 100 # Maximum number of allowable connections, or None for no limits. The default is 100.
134+ pool_maxsize : 10 # Number of allowable keep-alive connections, or None to always allow. The default is 10.
135+ enable_http2: True # See https://www.python-httpx.org/http2/
136+ # uncomment below section if you want to use a proxy
137+ # proxies:
138+ # all://:
139+ # - http://proxy1:8080
140+ # - http://proxy2:8080
141+ # uncomment below section only if you have more than one network interface
142+ # which can be the source of outgoing search requests
143+ # source_ips:
144+ # - 1.1.1.1
145+ # - 1.1.1.2
146+ # - fe80::/126
147+
148+
149+``request_timeout`` :
150+ Global timeout of the requests made to others engines in seconds. A bigger
151+ timeout will allow to wait for answers from slow engines, but in consequence
152+ will slow searx reactivity (the result page may take the time specified in the
153+ timeout to load). Can be override by :ref:`settings engine`
109154
110−``outgoing_proxies`` :
111− Define a proxy you wish to use, see `requests proxies`_. SOCKS proxies are
112− not supported / see `PR SOCKS support`.
155+``useragent_suffix`` :
156+ Suffix to the user-agent searx uses to send requests to others engines. If an
157+ engine wish to block you, a contact info here may be useful to avoid that.
158+
159+``keepalive_expiry``:
160+ Number of seconds to keep a connection in the pool. By default 5.0 seconds.
161+
162+.. _httpx proxies: https://www.python-httpx.org/advanced/#http-proxying
163+
164+``proxies`` :
165+ Define one or more proxies you wish to use, see `httpx proxies`_.
166+ If there are more than one proxy for one protocol (http, https),
167+ requests to the engines are distributed in a round-robin fashion.
113168
114169``source_ips`` :
115170 If you use multiple network interfaces, define from which IP the requests must
116− be made.
171+ be made. Example:
172+
173+ * ``0.0.0.0`` any local IPv4 address.
174+ * ``::`` any local IPv6 address.
175+ * ``192.168.0.1``
176+ * ``[ 192.168.0.1, 192.168.0.2 ]`` these two specific IP addresses
177+ * ``fe80::60a2:1691:e5a2:ee1f``
178+ * ``fe80::60a2:1691:e5a2:ee1f/126`` all IP addresses in this network.
179+ * ``[ 192.168.0.1, fe80::/126 ]``
180+
181+``retries`` :
182+ Number of retry in case of an HTTP error.
183+ On each retry, searx uses an different proxy and source ip.
184+
185+``retry_on_http_error`` :
186+ Retry request on some HTTP status code.
187+
188+ Example:
189+
190+ * ``true`` : on HTTP status code between 400 and 599.
191+ * ``403`` : on HTTP status code 403.
192+ * ``[403, 429]``: on HTTP status code 403 and 429.
193+
194+``enable_http2`` :
195+ Enable by default. Set to ``False`` to disable HTTP/2.
196+
197+``max_redirects`` :
198+ 30 by default. Maximum redirect before it is an error.
199+
200+
201+``locales:``
202+------------
203+
204+.. code:: yaml
205+
206+ locales:
207+ en : English
208+ de : Deutsch
209+ he : Hebrew
210+ hu : Magyar
211+ fr : Français
212+ es : Español
213+ it : Italiano
214+ nl : Nederlands
215+ ja : 日本語 (Japanese)
216+ tr : Türkçe
217+ ru : Russian
218+ ro : Romanian
117219
118220``locales`` :
119221 Locales codes and their names. Available translations of searx interface.
@@ -139,6 +241,22 @@ Engine settings
139241 api_key : 'apikey'
140242 disabled : True
141243 language : en_US
244+ #enable_http: False
245+ #enable_http2: False
246+ #retries: 1
247+ #retry_on_http_error: True # or 403 or [404, 429]
248+ #max_connections: 100
249+ #max_keepalive_connections: 10
250+ #keepalive_expiry: 5.0
251+ #proxies:
252+ # http:
253+ # - http://proxy1:8080
254+ # - http://proxy2:8080
255+ # https:
256+ # - http://proxy1:8080
257+ # - http://proxy2:8080
258+ # - socks5://user:password@proxy3:1080
259+ # - socks5h://user:password@proxy4:1080
142260
143261``name`` :
144262 Name that will be used across searx to define this engine. In settings, on
@@ -184,7 +302,93 @@ Engine settings
184302``display_error_messages`` : default ``True``
185303 When an engine returns an error, the message is displayed on the user interface.
186304
305+``network``: optional
306+ Use the network configuration from another engine.
307+ In addition, there are two default networks:
308+ * ``ipv4`` set ``local_addresses`` to ``0.0.0.0`` (use only IPv4 local addresses)
309+ * ``ipv6`` set ``local_addresses`` to ``::`` (use only IPv6 local addresses)
310+
187311.. note::
188312
189313 A few more options are possible, but they are pretty specific to some
190314 engines, and so won't be described here.
315+
316+
317+.. _settings use_default_settings:
318+
319+use_default_settings
320+====================
321+
322+.. sidebar:: ``use_default_settings: True``
323+
324+ - :ref:`settings location`
325+ - :ref:`use_default_settings.yml`
326+ - :origin:`/etc/searx/settings.yml <utils/templates/etc/searx/use_default_settings.yml>`
327+
328+The user defined ``settings.yml`` is loaded from the :ref:`settings location`
329+and can relied on the default configuration :origin:`searx/settings.yml` using:
330+
331+ ``use_default_settings: True``
332+
333+``server:``
334+ In the following example, the actual settings are the default settings defined
335+ in :origin:`searx/settings.yml` with the exception of the ``secret_key`` and
336+ the ``bind_address``:
337+
338+ .. code-block:: yaml
339+
340+ use_default_settings: True
341+ server:
342+ secret_key: "uvys6bRhKHUdFF5CqbJonSDSRN8H0sCBziNSrDGNVdpz7IeZhveVart3yvghoKHA"
343+ bind_address: "0.0.0.0"
344+
345+``engines:``
346+ With ``use_default_settings: True``, each settings can be override in a
347+ similar way, the ``engines`` section is merged according to the engine
348+ ``name``. In this example, searx will load all the engine and the arch linux
349+ wiki engine has a :ref:`token<private engines>`:
350+
351+ .. code-block:: yaml
352+
353+ use_default_settings: True
354+ server:
355+ secret_key: "uvys6bRhKHUdFF5CqbJonSDSRN8H0sCBziNSrDGNVdpz7IeZhveVart3yvghoKHA"
356+ engines:
357+ - name: arch linux wiki
358+ tokens: ['$ecretValue']
359+
360+``engines:`` / ``remove:``
361+ It is possible to remove some engines from the default settings. The following
362+ example is similar to the above one, but searx doesn't load the the google
363+ engine:
364+
365+ .. code-block:: yaml
366+
367+ use_default_settings:
368+ engines:
369+ remove:
370+ - google
371+ server:
372+ secret_key: "uvys6bRhKHUdFF5CqbJonSDSRN8H0sCBziNSrDGNVdpz7IeZhveVart3yvghoKHA"
373+ engines:
374+ - name: arch linux wiki
375+ tokens: ['$ecretValue']
376+
377+``engines:`` / ``keep_only:``
378+ As an alternative, it is possible to specify the engines to keep. In the
379+ following example, searx has only two engines:
380+
381+ .. code-block:: yaml
382+
383+ use_default_settings:
384+ engines:
385+ keep_only:
386+ - google
387+ - duckduckgo
388+ server:
389+ secret_key: "uvys6bRhKHUdFF5CqbJonSDSRN8H0sCBziNSrDGNVdpz7IeZhveVart3yvghoKHA"
390+ engines:
391+ - name: google
392+ tokens: ['$ecretValue']
393+ - name: duckduckgo
394+ tokens: ['$ecretValue']
docs/admin/update-searx.rst+46−10
@@ -4,20 +4,56 @@
44How to update
55=============
66
7+How to update depends on the :ref:`installation` method. If you have used the
8+:ref:`installation scripts`, use ``update`` command from the scripts.
9+
10+**Update** :ref:`searx service <searx.sh>`
11+
12+.. code:: sh
13+
14+ sudo -H ./utils/searx.sh update searx
15+
16+**Update** :ref:`filtron reverse proxy <filtron.sh>`
17+
718.. code:: sh
819
9− sudo -H -u searx -i
10− (searx)$ git stash
11− (searx)$ git pull origin master
12− (searx)$ git stash apply
13− (searx)$ ./manage.sh update_packages
20+ sudo -H ./utils/filtron.sh update filtron
21+
22+**Update** :ref:`result proxy <morty.sh>`
23+
24+.. code:: bash
25+
26+ sudo -H ./utils/morty.sh update morty
27+
28+.. _inspect searx:
29+
30+======================
31+How to inspect & debug
32+======================
33+
34+.. sidebar:: further read
35+
36+ - :ref:`toolboxing`
37+ - :ref:`Makefile`
38+
39+How to debug depends on the :ref:`installation` method. If you have used the
40+:ref:`installation scripts`, use ``inspect`` command from the scripts.
41+
42+**Inspect** :ref:`searx service <searx.sh>`
43+
44+.. code:: sh
45+
46+ sudo -H ./utils/searx.sh inspect service
47+
48+**Inspect** :ref:`filtron reverse proxy <filtron.sh>`
49+
50+.. code:: sh
1451
15−Restart uwsgi:
52+ sudo -H ./utils/filtron.sh inspect service
1653
17−.. tabs::
54+**Inspect** :ref:`result proxy <morty.sh>`
1855
19− .. group-tab:: Ubuntu / debian
56+.. code:: bash
2057
21− .. code:: sh
58+ sudo -H ./utils/morty.sh inspect service
2259
23− sudo -H systemctl restart uwsgi
docs/blog/index.rst+6−3
@@ -3,12 +3,15 @@ Blog
33====
44
55.. toctree::
6− :maxdepth: 2
7− :caption: Contents
6+ :titlesonly:
7+ :reversed:
88
9− lxcdev-202006
109 python3
1110 admin
1211 intro-offline
1312 private-engines
13+ lxcdev-202006
1414 command-line-engines
15+ search-indexer-engines
16+ sql-engines
17+ search-database-engines
docs/blog/lxcdev-202006.rst+5−5
@@ -31,7 +31,7 @@ might fail in some aspects we should not overlook.
3131
3232 The environment in which we run all our development processes matters!
3333
34−The :ref:`makefile` and the :ref:`make pyenv` encapsulate a lot for us, but they
34+The :ref:`makefile` and the :ref:`make install` encapsulate a lot for us, but they
3535do not have access to all prerequisites. For example, there may have
3636dependencies on packages that are installed on the developer's desktop, but
3737usually are not preinstalled on a server or client system. Another examples
@@ -259,8 +259,8 @@ suite. For this, we have to keep an eye on the :ref:`installation basic`:
259259- virtualenv in: ``/usr/local/searx/searx-pyenv``
260260- searx software in: ``/usr/local/searx/searx-src``
261261
262−The searx software is a clone of the ``GIT_URL`` (see :ref:`makefile setup`) and
263−the working tree is checked out from the ``GIT_BRANCH``. With the use of the
262+The searx software is a clone of the ``git_url`` (see :ref:`settings global`) and
263+the working tree is checked out from the ``git_branch``. With the use of the
264264:ref:`searx.sh` the searx service was installed as :ref:`uWSGI application
265265<searx uwsgi>`. To maintain this service, we can use ``systemctl`` (compare
266266:ref:`service architectures on distributions <uwsgi configuration>`).
@@ -356,7 +356,7 @@ daily usage:
356356 .. code:: sh
357357
358358 $ sudo -H ./utils/lxc.sh cmd searx-archlinux \
359− make docs
359+ make docs.html
360360
361361.. _blog-lxcdev-202006 abstract:
362362
@@ -407,7 +407,7 @@ To get remarks from the suite of the archlinux container we can use:
407407 ...
408408 [searx-archlinux] INFO: (eth0) filtron: http://10.174.184.156:4004/ http://10.174.184.156/searx
409409 [searx-archlinux] INFO: (eth0) morty: http://10.174.184.156:3000/
410− [searx-archlinux] INFO: (eth0) docs-live: http://10.174.184.156:8080/
410+ [searx-archlinux] INFO: (eth0) docs.live: http://10.174.184.156:8080/
411411 [searx-archlinux] INFO: (eth0) IPv6: http://[fd42:573b:e0b3:e97e:216:3eff:fea5:9b65]
412412 ...
413413
docs/blog/private-engines.rst+2
@@ -7,6 +7,8 @@ enabled engines on their instances. It might be because they do not want to
77expose some private information through an offline engine. Or they
88would rather share engines only with their trusted friends or colleagues.
99
10+.. _private engines:
11+
1012Private engines
1113===============
1214
docs/blog/search-database-engines.rstnew file+95
@@ -0,0 +1,95 @@
1+===============================
2+Query more of your NoSQL stores
3+===============================
4+
5+From now on, searx lets you to query your NoSQL data stores:
6+
7+* `Redis`_
8+* `MongoDB`_
9+
10+The reference configuration of the engines are included ``settings.yml`` just commented out,
11+as you have to set various options and install dependencies before using them.
12+
13+By default, the engines use ``key-value`` template for displaying results.
14+If you are not satisfied with the original result layout,
15+you can use your owm template by placing the template under
16+``searx/templates/{theme_name}/result_templates/{template_name}`` and setting
17+``result_template`` attribute to ``{template_name}``.
18+
19+Futhermore, if you do not want to expose these engines on a public instance, you can
20+still add them and limit the access by setting ``tokens`` as described in the `blog post about
21+private engines`_.
22+
23+Configuring searx to use the stores
24+===================================
25+
26+NoSQL data stores are used for storing arbitrary data without first defining their
27+structure.
28+
29+Redis
30+-----
31+
32+Reqired package: ``redis``
33+
34+Redis is a key value based data store usually stored in memory.
35+
36+Select a database to search in and set its index in the option ``db``. You can
37+either look for exact matches or use partial keywords to find what you are looking for
38+by configuring ``exact_match_only``.
39+
40+In this example you can search for exact matches in your first database:
41+
42+.. code:: yaml
43+
44+ - name : mystore
45+ engine : redis_server
46+ exact_match_only : True
47+ host : 127.0.0.1
48+ port : 6379
49+ password : secret-password
50+ db : 0
51+ shortcut : rds
52+ enable_http : True
53+
54+
55+MongoDB
56+-------
57+
58+Required package: ``pymongo``
59+
60+MongoDB is a document based database program that handles JSON like data.
61+
62+In order to query MongoDB, you have to select a ``database`` and a ``collection``. Furthermore,
63+you have to select a ``key`` that is going to be searched. MongoDB also supports the option ``exact_match_only``, so configure it
64+as you wish.
65+
66+Above is an example configuration for using a MongoDB collection:
67+
68+.. code:: yaml
69+
70+ - name : mymongo
71+ engine : mongodb
72+ shortcut : md
73+ host : '127.0.0.1'
74+ port : 27017
75+ database : personal
76+ collection : income
77+ key : month
78+ enable_http: True
79+
80+
81+Acknowledgement
82+===============
83+
84+This development was sponsored by `Search and Discovery Fund`_ of `NLnet Foundation`_ .
85+
86+.. _Redis: https://redis.io/
87+.. _MongoDB: https://mongodb.com/
88+.. _blog post about private engines: private-engines.html#private-engines
89+.. _Search and Discovery Fund: https://nlnet.nl/discovery
90+.. _NLnet Foundation: https://nlnet.nl/
91+
92+
93+| Happy hacking.
94+| kvch // 2021.07.13 23:16
95+
docs/blog/search-indexer-engines.rstnew file+114
@@ -0,0 +1,114 @@
1+===============================
2+Query your local search engines
3+===============================
4+
5+From now on, searx lets you to query your locally running search engines. The following
6+ones are supported now:
7+
8+* `Elasticsearch`_
9+* `Meilisearch`_
10+* `Solr`_
11+
12+All of the engines above are added to ``settings.yml`` just commented out, as you have to
13+``base_url`` for all them.
14+
15+Please note that if you are not using HTTPS to access these engines, you have to enable
16+HTTP requests by setting ``enable_http`` to ``True``.
17+
18+Futhermore, if you do not want to expose these engines on a public instance, you can
19+still add them and limit the access by setting ``tokens`` as described in the `blog post about
20+private engines`_.
21+
22+Configuring searx for search engines
23+====================================
24+
25+Each search engine is powerful, capable of full-text search.
26+
27+Elasticsearch
28+-------------
29+
30+Elasticsearch supports numerous ways to query the data it is storing. At the moment
31+the engine supports the most popular search methods: ``match``, ``simple_query_string``, ``term`` and ``terms``.
32+
33+If none of the methods fit your use case, you can select ``custom`` query type and provide the JSON payload
34+searx has to submit to Elasticsearch in ``custom_query_json``.
35+
36+The following is an example configuration for an Elasticsearch instance with authentication
37+configured to read from ``my-index`` index.
38+
39+.. code:: yaml
40+
41+ - name : elasticsearch
42+ shortcut : es
43+ engine : elasticsearch
44+ base_url : http://localhost:9200
45+ username : elastic
46+ password : changeme
47+ index : my-index
48+ query_type : match
49+ enable_http : True
50+
51+
52+Meilisearch
53+-----------
54+
55+This search engine is aimed at individuals and small companies. It is designed for
56+small-scale (less than 10 million documents) data collections. E.g. it is great for storing
57+web pages you have visited and searching in the contents later.
58+
59+The engine supports faceted search, so you can search in a subset of documents of the collection.
60+Futhermore, you can search in Meilisearch instances that require authentication by setting ``auth_token``.
61+
62+Here is a simple example to query a Meilisearch instance:
63+
64+.. code:: yaml
65+
66+ - name : meilisearch
67+ engine : meilisearch
68+ shortcut: mes
69+ base_url : http://localhost:7700
70+ index : my-index
71+ enable_http: True
72+
73+
74+Solr
75+----
76+
77+Solr is a popular search engine based on Lucene, just like Elasticsearch.
78+But instead of searching in indices, you can search in collections.
79+
80+This is an example configuration for searching in the collection ``my-collection`` and get
81+the results in ascending order.
82+
83+.. code:: yaml
84+
85+ - name : solr
86+ engine : solr
87+ shortcut : slr
88+ base_url : http://localhost:8983
89+ collection : my-collection
90+ sort : asc
91+ enable_http : True
92+
93+
94+Next steps
95+==========
96+
97+The next step is to add support for various SQL databases.
98+
99+Acknowledgement
100+===============
101+
102+This development was sponsored by `Search and Discovery Fund`_ of `NLnet Foundation`_ .
103+
104+.. _blog post about private engines: private-engines.html#private-engines
105+.. _Elasticsearch: https://www.elastic.co/elasticsearch/
106+.. _Meilisearch: https://www.meilisearch.com/
107+.. _Solr: https://solr.apache.org/
108+.. _Search and Discovery Fund: https://nlnet.nl/discovery
109+.. _NLnet Foundation: https://nlnet.nl/
110+
111+
112+| Happy hacking.
113+| kvch // 2021.04.07 23:16
114+
docs/blog/sql-engines.rstnew file+117
@@ -0,0 +1,117 @@
1+=================
2+Query SQL servers
3+=================
4+
5+Now you can query SQL servers using searx. The following ones are supported:
6+
7+* `PostgreSQL`_
8+* `MySQL`_
9+* `SQLite`_
10+
11+All of the engines above are added to ``settings.yml`` just commented out, as you have to
12+set the required attributes for the engines, e.g. ``database``. By default, the engines use
13+``key-value`` template for displaying results. If you are not satisfied with the original result layout,
14+you can use your owm template by placing the template under
15+``searx/templates/{theme_name}/result_templates/{template_name}`` and setting
16+``result_template`` attribute to ``{template_name}``.
17+
18+As mentioned in previous blog posts, if you do not wish to expose these engines on a
19+public instance, you can still add them and limit the access by setting ``tokens``
20+as described in the `blog post about private engines`_.
21+
22+Configure the engines
23+=====================
24+
25+The configuration of the new database engines are similar. You must put a valid
26+SELECT SQL query in ``query_str``. At the moment you can only bind at most
27+one parameter in your query. By setting the attribute ``limit`` you can
28+define how many results you want from the SQL server. Basically, it
29+is the same as the LIMIT keyword in SQL.
30+
31+Please, do not include LIMIT or OFFSET in your SQL query as the engines
32+rely on these keywords during paging. If you want to configure the number of returned results
33+use the option ``limit``.
34+
35+PostgreSQL
36+----------
37+
38+PostgreSQL is a powerful and robust open source database.
39+
40+Before configuring the PostgreSQL engine, you must install the dependency ``psychopg2``.
41+
42+You can find an example configuration below:
43+
44+.. code:: yaml
45+
46+ - name : postgresql
47+ engine : postgresql
48+ database : my_database
49+ username : searx
50+ password : password
51+ query_str : 'SELECT * from my_table WHERE my_column = %(query)s'
52+ shortcut : psql
53+
54+
55+MySQL
56+-----
57+
58+MySQL is said to be the most popular open source database.
59+
60+Before enabling MySQL engine, you must install the package ``mysql-connector-python``.
61+
62+The authentication plugin is configurable by setting ``auth_plugin`` in the attributes.
63+By default it is set to ``caching_sha2_password``.
64+
65+This is an example configuration for quering a MySQL server:
66+
67+.. code:: yaml
68+
69+ - name : mysql
70+ engine : mysql_server
71+ database : my_database
72+ username : searx
73+ password : password
74+ limit : 5
75+ query_str : 'SELECT * from my_table WHERE my_column=%(query)s'
76+ shortcut : mysql
77+
78+
79+SQLite
80+------
81+
82+SQLite is a small, fast and reliable SQL database engine. It does not require
83+any extra dependency.
84+
85+You can read from your database ``my_database`` using this example configuration:
86+
87+.. code:: yaml
88+
89+ - name : sqlite
90+ engine : sqlite
91+ shortcut: sq
92+ database : my_database
93+ query_str : 'SELECT * FROM my_table WHERE my_column=:query'
94+
95+
96+Next steps
97+==========
98+
99+The next step is to add support for more data stores, e.g. Redis and MongoDB.
100+
101+Acknowledgement
102+===============
103+
104+This development was sponsored by `Search and Discovery Fund`_ of `NLnet Foundation`_ .
105+
106+.. _PostgreSQL: https://www.postgresql.org/
107+.. _MySQL: https://www.mysql.com/
108+.. _SQLite: https://www.sqlite.org/index.html
109+.. _blog post about private engines: private-engines.html#private-engines
110+.. _Search and Discovery Fund: https://nlnet.nl/discovery
111+.. _NLnet Foundation: https://nlnet.nl/
112+
113+
114+| Happy hacking.
115+| kvch // 2021.05.23 23:16
116+
117+
docs/build-templates/morty.rst+1−1
@@ -32,7 +32,7 @@
3232 (${SERVICE_USER}) $ mkdir ${SERVICE_HOME}/local
3333 (${SERVICE_USER}) $ wget --progress=bar -O \"${GO_TAR}\" \\
3434 \"${GO_PKG_URL}\"
35− (${SERVICE_USER}) $ tar -C ${SERVICE_HOME}/local/go -xzf \"${GO_TAR}\"
35+ (${SERVICE_USER}) $ tar -C ${SERVICE_HOME}/local -xzf \"${GO_TAR}\"
3636 (${SERVICE_USER}) $ which go
3737 ${SERVICE_HOME}/local/go/bin/go
3838
docs/build-templates/searx.rst+19−2
@@ -116,6 +116,7 @@ ${fedora_build}
116116 pip install -U pip
117117 pip install -U setuptools
118118 pip install -U wheel
119+ pip install -U pyyaml
119120
120121 # jump to searx's working tree and install searx into virtualenv
121122 (${SERVICE_USER})$ cd \"$SEARX_SRC\"
@@ -128,12 +129,28 @@ ${fedora_build}
128129
129130.. tabs::
130131
131− .. group-tab:: bash
132+ .. group-tab:: Use default settings
133+
134+ .. code-block:: sh
135+
136+ $ sudo -H mkdir -p \"$(dirname ${SEARX_SETTINGS_PATH})\"
137+ $ sudo -H cp \"$SEARX_SRC/utils/templates/etc/searx/use_default_settings.yml\" \\
138+ \"${SEARX_SETTINGS_PATH}\"
139+
140+ .. group-tab:: searx/settings.yml
132141
133142 .. code-block:: sh
134143
135144 $ sudo -H mkdir -p \"$(dirname ${SEARX_SETTINGS_PATH})\"
136− $ sudo -H cp \"$SEARX_SRC/searx/settings.yml\" \"${SEARX_SETTINGS_PATH}\"
145+ $ sudo -H cp \"$SEARX_SRC/searx/settings.yml\" \\
146+ \"${SEARX_SETTINGS_PATH}\"
147+
148+.. tabs::
149+
150+ .. group-tab:: minimal setup
151+
152+ .. code-block:: sh
153+
137154 $ sudo -H sed -i -e \"s/ultrasecretkey/\$(openssl rand -hex 16)/g\" \"$SEARX_SETTINGS_PATH\"
138155 $ sudo -H sed -i -e \"s/{instance_name}/searx@\$(uname -n)/g\" \"$SEARX_SETTINGS_PATH\"
139156
docs/conf.py+35−31
@@ -1,20 +1,16 @@
11# -*- coding: utf-8 -*-
2+# SPDX-License-Identifier: AGPL-3.0-or-later
23
34import sys, os
4−from sphinx_build_tools import load_sphinx_config
5−from searx.version import VERSION_STRING
65from pallets_sphinx_themes import ProjectLink
76
8−from searx.brand import GIT_URL
9−GIT_BRANCH = os.environ.get("GIT_BRANCH", "master")
10−from searx.brand import SEARX_URL
11−from searx.brand import DOCS_URL
12−
7+from searx import brand
8+from searx.version import VERSION_STRING
139
1410# Project --------------------------------------------------------------
1511
1612project = u'searx'
17−copyright = u'2015-2020, Adam Tauber, Noémi Ványi'
13+copyright = u'2015-2021, Adam Tauber, Noémi Ványi'
1814author = u'Adam Tauber'
1915release, version = VERSION_STRING, VERSION_STRING
2016highlight_language = 'none'
@@ -27,9 +23,15 @@ numfig = True
2723
2824exclude_patterns = ['build-templates/*.rst']
2925
30−from searx import webapp
26+import searx.search
27+import searx.engines
28+import searx.plugins
29+searx.search.initialize()
3130jinja_contexts = {
32− 'webapp': dict(**webapp.__dict__),
31+ 'searx': {
32+ 'engines': searx.engines.engines,
33+ 'plugins': searx.plugins.plugins
34+ },
3335}
3436
3537# usage:: lorem :patch:`f373169` ipsum
@@ -40,20 +42,20 @@ extlinks['wiki'] = ('https://github.com/searx/searx/wiki/%s', ' ')
4042extlinks['pull'] = ('https://github.com/searx/searx/pull/%s', 'PR ')
4143
4244# links to custom brand
43−extlinks['origin'] = (GIT_URL + '/blob/' + GIT_BRANCH + '/%s', 'git://')
44−extlinks['patch'] = (GIT_URL + '/commit/%s', '#')
45−extlinks['search'] = (SEARX_URL + '/%s', '#')
46−extlinks['docs'] = (DOCS_URL + '/%s', 'docs: ')
45+extlinks['origin'] = (brand.GIT_URL + '/blob/' + brand.GIT_BRANCH + '/%s', 'git://')
46+extlinks['patch'] = (brand.GIT_URL + '/commit/%s', '#')
47+extlinks['search'] = (brand.SEARX_URL + '/%s', '#')
48+extlinks['docs'] = (brand.DOCS_URL + '/%s', 'docs: ')
4749extlinks['pypi'] = ('https://pypi.org/project/%s', 'PyPi: ')
4850extlinks['man'] = ('https://manpages.debian.org/jump?q=%s', '')
4951#extlinks['role'] = (
5052# 'https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-%s', '')
5153extlinks['duref'] = (
52− 'http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#%s', '')
54+ 'https://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#%s', '')
5355extlinks['durole'] = (
54− 'http://docutils.sourceforge.net/docs/ref/rst/roles.html#%s', '')
56+ 'https://docutils.sourceforge.net/docs/ref/rst/roles.html#%s', '')
5557extlinks['dudir'] = (
56− 'http://docutils.sourceforge.net/docs/ref/rst/directives.html#%s', '')
58+ 'https://docutils.sourceforge.net/docs/ref/rst/directives.html#%s', '')
5759extlinks['ctan'] = (
5860 'https://ctan.org/pkg/%s', 'CTAN: ')
5961
@@ -87,6 +89,7 @@ issues_github_path = "searx/searx"
8789# HTML -----------------------------------------------------------------
8890
8991sys.path.append(os.path.abspath('_themes'))
92+sys.path.insert(0, os.path.abspath("../utils/"))
9093html_theme_path = ['_themes']
9194html_theme = "searx"
9295
@@ -97,14 +100,21 @@ imgmath_font_size = 14
97100# sphinx.ext.imgmath setup END
98101
99102html_theme_options = {"index_sidebar_logo": True}
100−html_context = {
101− "project_links": [
102− ProjectLink("Source", GIT_URL),
103− ProjectLink("Wiki", "https://github.com/searx/searx/wiki"),
104− ProjectLink("Public instances", "https://searx.space/"),
105− ProjectLink("Twitter", "https://twitter.com/Searx_engine"),
106− ]
107−}
103+html_context = {"project_links": [] }
104+html_context["project_links"].append(ProjectLink("Blog", "blog/index.html"))
105+if brand.GIT_URL:
106+ html_context["project_links"].append(ProjectLink("Source", brand.GIT_URL))
107+if brand.WIKI_URL:
108+ html_context["project_links"].append(ProjectLink("Wiki", brand.WIKI_URL))
109+if brand.PUBLIC_INSTANCES:
110+ html_context["project_links"].append(ProjectLink("Public instances", brand.PUBLIC_INSTANCES))
111+if brand.TWITTER_URL:
112+ html_context["project_links"].append(ProjectLink("Twitter", brand.TWITTER_URL))
113+if brand.ISSUE_URL:
114+ html_context["project_links"].append(ProjectLink("Issue Tracker", brand.ISSUE_URL))
115+if brand.CONTACT_URL:
116+ html_context["project_links"].append(ProjectLink("Contact", brand.CONTACT_URL))
117+
108118html_sidebars = {
109119 "**": ["project.html", "relations.html", "searchbox.html"],
110120}
@@ -119,9 +129,3 @@ html_show_sourcelink = False
119129latex_documents = [
120130 (master_doc, "searx-{}.tex".format(VERSION_STRING), html_title, author, "manual")
121131]
122−
123−# ------------------------------------------------------------------------------
124−# Since loadConfig overwrites settings from the global namespace, it has to be
125−# the last statement in the conf.py file
126−# ------------------------------------------------------------------------------
127−load_sphinx_config(globals())
docs/dev/contribution_guide.rst+14−25
@@ -87,7 +87,7 @@ In order to submit a patch, please follow the steps below:
8787
8888- Add yourself to the :origin:`AUTHORS.rst` file.
8989
90−- Choose meaning full commit messages, read `Conventional Commits`_
90+- Choose meaningful commit messages, read `Conventional Commits`_
9191
9292 .. code::
9393
@@ -117,8 +117,8 @@ Translation currently takes place on :ref:`transifex <translation>`.
117117Documentation
118118=============
119119
120−.. _Sphinx: http://www.sphinx-doc.org
121−.. _reST: http://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
120+.. _Sphinx: https://www.sphinx-doc.org
121+.. _reST: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
122122
123123.. sidebar:: The reST sources
124124
@@ -132,11 +132,11 @@ Here is an example which makes a complete rebuild:
132132
133133.. code:: sh
134134
135− $ make docs-clean docs
135+ $ make docs.clean docs.html
136136 ...
137137 The HTML pages are in dist/docs.
138138
139−.. _make docs-live:
139+.. _make docs.live:
140140
141141live build
142142----------
@@ -144,19 +144,19 @@ live build
144144.. _sphinx-autobuild:
145145 https://github.com/executablebooks/sphinx-autobuild/blob/master/README.md
146146
147−.. sidebar:: docs-clean
147+.. sidebar:: docs.clean
148148
149149 It is recommended to assert a complete rebuild before deploying (use
150− ``docs-clean``).
150+ ``docs.clean``).
151151
152152Live build is like WYSIWYG. If you want to edit the documentation, its
153−recommended to use. The Makefile target ``docs-live`` builds the docs, opens
153+recommended to use. The Makefile target ``docs.live`` builds the docs, opens
154154URL in your favorite browser and rebuilds every time a reST file has been
155155changed.
156156
157157.. code:: sh
158158
159− $ make docs-live
159+ $ make docs.live
160160 ...
161161 The HTML pages are in dist/docs.
162162 ... Serving on http://0.0.0.0:8000
@@ -169,7 +169,7 @@ argument. E.g to find and use a free port, use:
169169
170170.. code:: sh
171171
172− $ SPHINXOPTS="--port 0" make docs-live
172+ $ SPHINXOPTS="--port 0" make docs.live
173173 ...
174174 ... Serving on http://0.0.0.0:50593
175175 ...
@@ -180,21 +180,10 @@ argument. E.g to find and use a free port, use:
180180deploy on github.io
181181-------------------
182182
183−To deploy documentation at :docs:`github.io <.>` use Makefile target
184−:ref:`make gh-pages`, which will builds the documentation, clones searx into a sub
185−folder ``gh-pages``, cleans it, copies the doc build into and runs all the
186−needed git add, commit and push:
183+To deploy documentation at :docs:`github.io <.>` use Makefile target :ref:`make
184+docs.gh-pages`, which builds the documentation and runs all the needed git add,
185+commit and push:
187186
188187.. code:: sh
189188
190− $ make docs-clean gh-pages
191− ...
192− SPHINX docs --> file://<...>/dist/docs
193− The HTML pages are in dist/docs.
194− ...
195− Cloning into 'gh-pages' ...
196− ...
197− cd gh-pages; git checkout gh-pages >/dev/null
198− Switched to a new branch 'gh-pages'
199− ...
200− doc available at --> https://searx.github.io/searx
189+ $ make docs.clean docs.gh-pages
docs/dev/engine_overview.rst+63−28
@@ -18,6 +18,9 @@ engines. Adapters are stored under the folder :origin:`searx/engines`.
1818 :depth: 3
1919 :backlinks: entry
2020
21+
22+.. _general engine configuration:
23+
2124general engine configuration
2225============================
2326
@@ -34,15 +37,15 @@ settings. However, the standard way is the following:
3437engine file
3538-----------
3639
37−======================= =========== ===========================================
40+======================= =========== ========================================================
3841argument type information
39−======================= =========== ===========================================
42+======================= =========== ========================================================
4043categories list pages, in which the engine is working
4144paging boolean support multible pages
42−language_support boolean support language choosing
4345time_range_support boolean support search time range
44−offline boolean engine runs offline
45−======================= =========== ===========================================
46+engine_type str ``online`` by default, other possibles values are
47+ ``offline``, ``online_dictionnary``, ``online_currency``
48+======================= =========== ========================================================
4649
4750.. _engine settings:
4851
@@ -55,6 +58,8 @@ argument type information
5558name string name of search-engine
5659engine string name of searx-engine
5760 (filename without ``.py``)
61+enable_http bool enable HTTP
62+ (by default only HTTPS is enabled).
5863shortcut string shortcut of search-engine
5964timeout string specific timeout for search-engine
6065display_error_messages boolean display error messages on the web UI
@@ -93,7 +98,6 @@ example code
9398 # engine dependent config
9499 categories = ['general']
95100 paging = True
96− language_support = True
97101
98102
99103making a request
@@ -108,22 +112,49 @@ passed arguments
108112These arguments can be used to construct the search query. Furthermore,
109113parameters with default value can be redefined for special purposes.
110114
115+If the ``engine_type`` is ``online```:
116+
117+====================== ============== ========================================================================
118+argument type default-value, information
119+====================== ============== ========================================================================
120+url str ``''``
121+method str ``'GET'``
122+headers set ``{}``
123+data set ``{}``
124+cookies set ``{}``
125+verify bool ``True``
126+headers.User-Agent str a random User-Agent
127+category str current category, like ``'general'``
128+safesearch int ``0``, between ``0`` and ``2`` (normal, moderate, strict)
129+time_range Optional[str] ``None``, can be ``day``, ``week``, ``month``, ``year``
130+pageno int current pagenumber
131+language str specific language code like ``'en_US'``, or ``'all'`` if unspecified
132+====================== ============== ========================================================================
133+
134+
135+If the ``engine_type`` is ``online_dictionnary```, in addition to the ``online`` arguments:
136+
137+====================== ============ ========================================================================
138+argument type default-value, information
139+====================== ============ ========================================================================
140+from_lang str specific language code like ``'en_US'``
141+to_lang str specific language code like ``'en_US'``
142+query str the text query without the languages
143+====================== ============ ========================================================================
144+
145+If the ``engine_type`` is ``online_currency```, in addition to the ``online`` arguments:
146+
111147====================== ============ ========================================================================
112148argument type default-value, information
113149====================== ============ ========================================================================
114−url string ``''``
115−method string ``'GET'``
116−headers set ``{}``
117−data set ``{}``
118−cookies set ``{}``
119−verify boolean ``True``
120−headers.User-Agent string a random User-Agent
121−category string current category, like ``'general'``
122−started datetime current date-time
123−pageno int current pagenumber
124−language string specific language code like ``'en_US'``, or ``'all'`` if unspecified
150+amount float the amount to convert
151+from str ISO 4217 code
152+to str ISO 4217 code
153+from_name str currency name
154+to_name str currency name
125155====================== ============ ========================================================================
126156
157+
127158parsed arguments
128159----------------
129160
@@ -131,16 +162,20 @@ The function ``def request(query, params):`` always returns the ``params``
131162variable. Inside searx, the following paramters can be used to specify a search
132163request:
133164
134−============ =========== =========================================================
135−argument type information
136−============ =========== =========================================================
137−url string requested url
138−method string HTTP request method
139−headers set HTTP header information
140−data set HTTP data information (parsed if ``method != 'GET'``)
141−cookies set HTTP cookies
142−verify boolean Performing SSL-Validity check
143−============ =========== =========================================================
165+=================== =========== ==========================================================================
166+argument type information
167+=================== =========== ==========================================================================
168+url str requested url
169+method str HTTP request method
170+headers set HTTP header information
171+data set HTTP data information
172+cookies set HTTP cookies
173+verify bool Performing SSL-Validity check
174+follow_redirects bool Follow redirects
175+max_redirects int maximum redirects, hard limit
176+soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine
177+raise_for_httperror bool True by default: raise an exception if the HTTP code of response is >= 300
178+=================== =========== ==========================================================================
144179
145180
146181example code
@@ -259,7 +294,7 @@ latitude latitude of result (in decimal format)
259294longitude longitude of result (in decimal format)
260295boundingbox boundingbox of result (array of 4. values
261296 ``[lat-min, lat-max, lon-min, lon-max]``)
262−geojson geojson of result (http://geojson.org)
297+geojson geojson of result (https://geojson.org/)
263298osm.type type of osm-object (if OSM-Result)
264299osm.id id of osm-object (if OSM-Result)
265300address.name name of object
docs/dev/makefile.rst+124−123
@@ -1,65 +1,33 @@
11.. _makefile:
22
3−================
4−Makefile Targets
5−================
3+========
4+Makefile
5+========
66
77.. _gnu-make: https://www.gnu.org/software/make/manual/make.html#Introduction
88
99.. sidebar:: build environment
1010
11− Before looking deeper at the targets, first read about :ref:`makefile setup`
12− and :ref:`make pyenv`.
11+ Before looking deeper at the targets, first read about :ref:`make
12+ install`.
1313
1414 To install system requirements follow :ref:`buildhosts`.
1515
16−With the aim to simplify development cycles, started with :pull:`1756` a
17−``Makefile`` based boilerplate was added. If you are not familiar with
18−Makefiles, we recommend to read gnu-make_ introduction.
16+All relevant build tasks are implemented in :origin:`manage.sh` and for CI or
17+IDE integration a small ``Makefile`` wrapper is available. If you are not
18+familiar with Makefiles, we recommend to read gnu-make_ introduction.
1919
2020The usage is simple, just type ``make {target-name}`` to *build* a target.
2121Calling the ``help`` target gives a first overview (``make help``):
2222
2323.. program-output:: bash -c "cd ..; make --no-print-directory help"
2424
25−
2625.. contents:: Contents
2726 :depth: 2
2827 :local:
2928 :backlinks: entry
3029
31−
32−.. _makefile setup:
33−
34−Makefile setup
35−==============
36−
37−.. _git stash: https://git-scm.com/docs/git-stash
38−
39−.. sidebar:: fork & upstream
40−
41− Commit changes in your (local) branch, fork or whatever, but do not push them
42− upstream / `git stash`_ is your friend.
43−
44−The main setup is done in the :origin:`Makefile`.
45−
46−.. literalinclude:: ../../Makefile
47− :start-after: START Makefile setup
48− :end-before: END Makefile setup
49−
50−:GIT_URL: Changes this, to point to your searx fork.
51−:GIT_BRANCH: Changes this, to point to your searx branch.
52−:SEARX_URL: Changes this, to point to your searx instance.
53−:DOCS_URL: If you host your own (*brand*) documentation, change this URL.
54−
55−If you change any of this build environment variables, you have to run ``make
56−buildenv``::
57−
58− $ make buildenv
59− build searx/brand.py
60− build utils/brand.env
61−
62−.. _make pyenv:
30+.. _make install:
6331
6432Python environment
6533==================
@@ -68,31 +36,42 @@ Python environment
6836
6937 ``source ./local/py3/bin/activate``
7038
71−With Makefile we do no longer need to build up the virualenv manually (as
72−described in the :ref:`devquickstart` guide). Jump into your git working tree
73−and release a ``make pyenv``:
74−
75−.. code:: sh
39+We do no longer need to build up the virtualenv manually. Jump into your git
40+working tree and release a ``make install`` to get a virtualenv with a
41+*developer install* of searx (:origin:`setup.py`). ::
7642
7743 $ cd ~/searx-clone
78− $ make pyenv
79− PYENV usage: source ./local/py3/bin/activate
44+ $ make install
45+ PYENV [virtualenv] installing ./requirements*.txt into local/py3
8046 ...
47+ PYENV OK
48+ PYENV [install] pip install -e 'searx[test]'
49+ ...
50+ Successfully installed argparse-1.4.0 searx
51+ BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
52+ BUILDENV INFO:searx:Initialisation done
53+ BUILDENV build utils/brand.env
8154
82−With target ``pyenv`` a development environment (aka virtualenv) was build up in
83−``./local/py3/``. To make a *developer install* of searx (:origin:`setup.py`)
84−into this environment, use make target ``install``:
85−
86−.. code:: sh
55+If you release ``make install`` multiple times the installation will only
56+rebuild if the sha256 sum of the *requirement files* fails. With other words:
57+the check fails if you edit the requirements listed in
58+:origin:`requirements-dev.txt` and :origin:`requirements.txt`). ::
8759
8860 $ make install
89− PYENV usage: source ./local/py3/bin/activate
90− PYENV using virtualenv from ./local/py3
91− PYENV install .
92−
93−You have never to think about intermediate targets like ``pyenv`` or
94−``install``, the ``Makefile`` chains them as requisites. Just run your main
95−target.
61+ PYENV OK
62+ PYENV [virtualenv] requirements.sha256 failed
63+ [virtualenv] - 6cea6eb6def9e14a18bf32f8a3e... ./requirements-dev.txt
64+ [virtualenv] - 471efef6c73558e391c3adb35f4... ./requirements.txt
65+ ...
66+ PYENV [virtualenv] installing ./requirements*.txt into local/py3
67+ ...
68+ PYENV OK
69+ PYENV [install] pip install -e 'searx[test]'
70+ ...
71+ Successfully installed argparse-1.4.0 searx
72+ BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
73+ BUILDENV INFO:searx:Initialisation done
74+ BUILDENV build utils/brand.env
9675
9776.. sidebar:: drop environment
9877
@@ -100,10 +79,7 @@ target.
10079 <make clean>` first.
10180
10281If you think, something goes wrong with your ./local environment or you change
103−the :origin:`setup.py` file (or the requirements listed in
104−:origin:`requirements-dev.txt` and :origin:`requirements.txt`), you have to call
105−:ref:`make clean`.
106−
82+the :origin:`setup.py` file, you have to call :ref:`make clean`.
10783
10884.. _make run:
10985
@@ -113,88 +89,113 @@ the :origin:`setup.py` file (or the requirements listed in
11389To get up a running a developer instance simply call ``make run``. This enables
11490*debug* option in :origin:`searx/settings.yml`, starts a ``./searx/webapp.py``
11591instance, disables *debug* option again and opens the URL in your favorite WEB
116−browser (:man:`xdg-open`):
117−
118−.. code:: sh
92+browser (:man:`xdg-open`)::
11993
120− $ make run
121− PYENV usage: source ./local/py3/bin/activate
122− PYENV install .
123− ./local/py3/bin/python ./searx/webapp.py
124− ...
125− INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
126− ...
94+ $ make run
95+ PYENV OK
96+ SEARX_DEBUG=1 ./manage.sh pyenv.cmd python ./searx/webapp.py
97+ ...
98+ INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
12799
128100.. _make clean:
129101
130102``make clean``
131103==============
132104
133−Drop all intermediate files, all builds, but keep sources untouched. Includes
134−target ``pyclean`` which drops ./local environment. Before calling ``make
135−clean`` stop all processes using :ref:`make pyenv`.
136−
137−.. code:: sh
105+Drop all intermediate files, all builds, but keep sources untouched. Before
106+calling ``make clean`` stop all processes using :ref:`make install`. ::
138107
139108 $ make clean
140− CLEAN pyclean
141− CLEAN clean
109+ CLEAN pyenv
110+ PYENV [virtualenv] drop ./local/py3
111+ CLEAN docs -- ./build/docs ./dist/docs
112+ CLEAN locally installed npm dependencies
113+ CLEAN test stuff
114+ CLEAN common files
142115
143116.. _make docs:
144117
145−``make docs docs-live docs-clean``
146−==================================
118+``make docs docs.autobuild docs.clean``
119+=======================================
147120
148−We describe the usage of the ``doc*`` targets in the :ref:`How to contribute /
121+We describe the usage of the ``doc.*`` targets in the :ref:`How to contribute /
149122Documentation <contrib docs>` section. If you want to edit the documentation
150−read our :ref:`make docs-live` section. If you are working in your own brand,
151−adjust your :ref:`Makefile setup <makefile setup>`.
152−
123+read our :ref:`make docs.live` section. If you are working in your own brand,
124+adjust your :ref:`settings global`.
153125
154−.. _make gh-pages:
126+.. _make docs.gh-pages:
155127
156−``make gh-pages``
157−=================
128+``make docs.gh-pages``
129+======================
158130
159−To deploy on github.io first adjust your :ref:`Makefile setup <makefile
160−setup>`. For any further read :ref:`deploy on github.io`.
131+To deploy on github.io first adjust your :ref:`settings global`. For any
132+further read :ref:`deploy on github.io`.
161133
162134.. _make test:
163135
164136``make test``
165137=============
166138
167−Runs a series of tests: ``test.pep8``, ``test.unit``, ``test.robot`` and does
168−additional :ref:`pylint checks <make pylint>`. You can run tests selective,
169−e.g.:
170−
171−.. code:: sh
139+Runs a series of tests: :ref:`make test.pylint`, ``test.pep8``, ``test.unit``
140+and ``test.robot``. You can run tests selective, e.g.::
172141
173142 $ make test.pep8 test.unit test.sh
174− . ./local/py3/bin/activate; ./manage.sh pep8_check
175− [!] Running pep8 check
176− . ./local/py3/bin/activate; ./manage.sh unit_tests
177− [!] Running unit tests
143+ TEST test.pep8 OK
144+ ...
145+ TEST test.unit OK
146+ ...
147+ TEST test.sh OK
148+
149+.. _make test.sh:
150+
151+``make test.sh``
152+================
153+
154+:ref:`sh lint` / if you have changed some bash scripting run this test before
155+commit.
178156
179−.. _make pylint:
157+.. _make test.pylint:
180158
181−``make pylint``
182−===============
159+``make test.pylint``
160+====================
183161
184162.. _Pylint: https://www.pylint.org/
185163
186−Before commiting its recommend to do some (more) linting. Pylint_ is known as
187−one of the best source-code, bug and quality checker for the Python programming
188−language. Pylint_ is not yet a quality gate within our searx project (like
189−:ref:`test.pep8 <make test>` it is), but Pylint_ can help to improve code
190−quality anyway. The pylint profile we use at searx project is found in
191−project's root folder :origin:`.pylintrc`.
164+Pylint_ is known as one of the best source-code, bug and quality checker for the
165+Python programming language. The pylint profile we use at searx project is
166+found in project's root folder :origin:`.pylintrc`.
192167
193−Code quality is a ongoing process. Don't try to fix all messages from Pylint,
194−run Pylint and check if your changed lines are bringing up new messages. If so,
195−fix it. By this, code quality gets incremental better and if there comes the
196−day, the linting is balanced out, we might decide to add Pylint as a quality
197−gate.
168+.. _make search.checker:
169+
170+``search.checker.{engine name}``
171+================================
172+
173+To check all engines::
174+
175+ make search.checker
176+
177+To check a engine with whitespace in the name like *google news* replace space
178+by underline::
179+
180+ make search.checker.google_news
181+
182+To see HTTP requests and more use SEARX_DEBUG::
183+
184+ make SEARX_DEBUG=1 search.checker.google_news
185+
186+.. _3xx: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_redirection
187+
188+To filter out HTTP redirects (3xx_)::
189+
190+ make SEARX_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]"
191+ ...
192+ Engine google news Checking
193+ https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
194+ https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
195+ --
196+ https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
197+ https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
198+ --
198199
199200
200201``make pybuild``
@@ -203,9 +204,7 @@ gate.
203204.. _PyPi: https://pypi.org/
204205.. _twine: https://twine.readthedocs.io/en/latest/
205206
206−Build Python packages in ``./dist/py``.
207−
208−.. code:: sh
207+Build Python packages in ``./dist/py``::
209208
210209 $ make pybuild
211210 ...
@@ -213,9 +212,11 @@ Build Python packages in ``./dist/py``.
213212 running sdist
214213 running egg_info
215214 ...
216− $ ls ./dist/py/
217− searx-0.15.0-py3-none-any.whl searx-0.15.0.tar.gz
215+ running bdist_wheel
216+
217+ $ ls ./dist
218+ searx-0.18.0-py3-none-any.whl searx-0.18.0.tar.gz
218219
219−To upload packages to PyPi_, there is also a ``upload-pypi`` target. It needs
220−twine_ to be installed. Since you are not the owner of :pypi:`searx` you will
221−never need the latter.
220+To upload packages to PyPi_, there is also a ``pypi.upload`` target (to test use
221+``pypi.upload.test``). Since you are not the owner of :pypi:`searx` you will
222+never need to upload.
docs/dev/quickstart.rst+25−102
@@ -4,129 +4,52 @@
44Development Quickstart
55======================
66
7−.. sidebar:: :ref:`makefile`
7+.. _npm: https://www.npmjs.com/
88
9− For additional developer purpose there are :ref:`makefile`.
10−
11−This quickstart guide gets your environment set up with searx. Furthermore, it
12−gives a short introduction to the ``manage.sh`` script.
13−
14−How to setup your development environment
15−=========================================
16−
17−.. sidebar:: :ref:`make pyenv <make pyenv>`
18−
19− Alternatively use the :ref:`make pyenv`.
20−
21−First, clone the source code of searx to the desired folder. In this case the
22−source is cloned to ``~/myprojects/searx``. Then create and activate the
23−searx-ve virtualenv and install the required packages using ``manage.sh``.
9+Searx loves developers, just clone and start hacking. All the rest is done for
10+you simply by using :ref:`make <makefile>`.
2411
2512.. code:: sh
2613
27− cd ~/myprojects
2814 git clone https://github.com/searx/searx.git
29− cd searx
30− python3 -m venv searx-ve
31− . ./searx-ve/bin/activate
32− ./manage.sh update_dev_packages
33−
34−
35−How to run tests
36−================
37−
38−.. sidebar:: :ref:`make test.unit <make test>`
39−
40− Alternatively use the ``test.pep8``, ``test.unit``, ``test.robot`` targets.
41−
42−Tests can be run using the ``manage.sh`` script. Following tests and checks are
43−available:
44−
45−- Unit tests
46−- Selenium tests
47−- PEP8 validation
48−- Unit test coverage check
49−
50−For example unit tests are run with the command below:
51−
52−.. code:: sh
53−
54− ./manage.sh unit_tests
55−
56−For further test options, please consult the help of the ``manage.sh`` script or
57−read :ref:`make test`.
58−
5915
60−How to compile styles and javascript
61−====================================
16+Here is how a minimal workflow looks like:
6217
63−.. _less: http://lesscss.org/
64−.. _NodeJS: https://nodejs.org
18+1. *start* hacking
19+2. *run* your code: :ref:`make run`
20+3. *test* your code: :ref:`make test`
6521
66−How to build styles
67−-------------------
22+If you think at some point something fails, go back to *start*. Otherwise,
23+choose a meaningful commit message and we are happy to receive your pull
24+request. To not end in *wild west* we have some directives, please pay attention
25+to our ":ref:`how to contribute`" guideline.
6826
69−Less_ is required to build the styles of searx. Less_ can be installed using
70−either NodeJS_ or Apt.
27+If you implement themes, you will need to compile styles and JavaScript before
28+*run*.
7129
7230.. code:: sh
7331
74− sudo -H apt-get install nodejs
75− sudo -H npm install -g less
76−
77−OR
78−
79−.. code:: sh
80−
81− sudo -H apt-get install node-less
82−
83−After satisfying the requirements styles can be build using ``manage.sh``
84−
85−.. code:: sh
86−
87− ./manage.sh styles
88−
89−
90−How to build the source of the themes
91−=====================================
92−
93−.. _grunt: https://gruntjs.com/
94−
95−Grunt_ must be installed in order to build the javascript sources. It depends on
96−NodeJS, so first Node has to be installed.
97−
98−.. code:: sh
32+ make themes
9933
100− sudo -H apt-get install nodejs
101− make node.env
34+Don't forget to install npm_ first.
10235
103−After installing grunt, the files can be built using the following command:
36+.. tabs::
10437
105−.. code:: sh
38+ .. group-tab:: Ubuntu / debian
10639
107− make themes
40+ .. code:: sh
10841
42+ sudo -H apt-get install npm
10943
110−Tips for debugging/development
111−==============================
44+ .. group-tab:: Arch Linux
11245
113−.. sidebar:: :ref:`make run`
46+ .. code-block:: sh
11447
115− Makefile target ``run`` already enables debug option for your developer
116− session / see :ref:`make run`.
48+ sudo -H pacman -S npm
11749
118−Turn on debug logging
119− Whether you are working on a new engine or trying to eliminate a bug, it is
120− always a good idea to turn on debug logging. When debug logging is enabled a
121− stack trace appears, instead of the cryptic ``Internal Server Error``
122− message. It can be turned on by setting ``debug: False`` to ``debug: True`` in
123− :origin:`settings.yml <searx/settings.yml>`.
50+ .. group-tab:: Fedora / RHEL
12451
125−.. sidebar:: :ref:`make test`
52+ .. code-block:: sh
12653
127− Alternatively use the :ref:`make test` targets.
54+ sudo -H dnf install npm
12855
129−Run ``./manage.sh tests`` before creating a PR.
130− Failing build on Travis is common because of PEP8 checks. So a new commit
131− must be created containing these format fixes. This phase can be skipped if
132− ``./manage.sh tests`` is run locally before creating a PR.
docs/dev/reST.rst+32−24
@@ -15,8 +15,8 @@ generated and deployed at :docs:`github.io <.>`. For build prerequisites read
1515:ref:`docs build`.
1616
1717The source files of Searx's documentation are located at :origin:`docs`. Sphinx
18−assumes source files to be encoded in UTF-8 by defaul. Run :ref:`make docs-live
19−<make docs-live>` to build HTML while editing.
18+assumes source files to be encoded in UTF-8 by defaul. Run :ref:`make docs.live
19+<make docs.live>` to build HTML while editing.
2020
2121.. sidebar:: Further reading
2222
@@ -319,6 +319,9 @@ To list all anchors of the inventory (e.g. ``python``) use:
319319.. code:: sh
320320
321321 $ python -m sphinx.ext.intersphinx https://docs.python.org/3/objects.inv
322+ ...
323+ $ python -m sphinx.ext.intersphinx https://searx.github.io/searx/objects.inv
324+ ...
322325
323326Literal blocks
324327==============
@@ -1273,28 +1276,33 @@ Templating
12731276
12741277.. sidebar:: Build environment
12751278
1276− All *generic-doc* tasks are running in the :ref:`build environment <make
1277− pyenv>`.
1279+ All *generic-doc* tasks are running in the :ref:`make install`.
12781280
12791281Templating is suitable for documentation which is created generic at the build
1280−time. The sphinx-jinja_ extension evaluates jinja_ templates in the :ref:`build
1281−environment <make pyenv>` (with searx modules installed). We use this e.g. to
1282−build chapter: :ref:`engines generic`. Below the jinja directive from the
1282+time. The sphinx-jinja_ extension evaluates jinja_ templates in the :ref:`make
1283+install` (with searx modules installed). We use this e.g. to build chapter:
1284+:ref:`engines generic`. Below the jinja directive from the
12831285:origin:`docs/admin/engines.rst` is shown:
12841286
12851287.. literalinclude:: ../admin/engines.rst
12861288 :language: reST
12871289 :start-after: .. _configured engines:
12881290
1289−The context for the template is selected in the line ``.. jinja:: webapp``. In
1290−sphinx's build configuration (:origin:`docs/conf.py`) the ``webapp`` context
1291−points to the name space of the python module: ``webapp``.
1291+The context for the template is selected in the line ``.. jinja:: searx``. In
1292+sphinx's build configuration (:origin:`docs/conf.py`) the ``searx`` context
1293+contains the ``engines`` and ``plugins``.
12921294
12931295.. code:: py
12941296
1295− from searx import webapp
1297+ import searx.search
1298+ import searx.engines
1299+ import searx.plugins
1300+ searx.search.initialize()
12961301 jinja_contexts = {
1297− 'webapp': dict(**webapp.__dict__)
1302+ 'searx': {
1303+ 'engines': searx.engines.engines,
1304+ 'plugins': searx.plugins.plugins
1305+ },
12981306 }
12991307
13001308
@@ -1391,27 +1399,27 @@ The next example shows the difference of ``\tfrac`` (*textstyle*) and ``\dfrac``
13911399
13921400.. _readability: https://docs.python-guide.org/writing/style/
13931401.. _Sphinx-Primer:
1394− http://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
1402+ https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
13951403.. _reST: https://docutils.sourceforge.io/rst.html
13961404.. _Sphinx Roles:
13971405 https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html
1398−.. _Sphinx: http://www.sphinx-doc.org
1399−.. _`sphinx-doc FAQ`: http://www.sphinx-doc.org/en/stable/faq.html
1406+.. _Sphinx: https://www.sphinx-doc.org
1407+.. _`sphinx-doc FAQ`: https://www.sphinx-doc.org/en/stable/faq.html
14001408.. _Sphinx markup constructs:
1401− http://www.sphinx-doc.org/en/stable/markup/index.html
1409+ https://www.sphinx-doc.org/en/stable/markup/index.html
14021410.. _`sphinx cross references`:
1403− http://www.sphinx-doc.org/en/stable/markup/inline.html#cross-referencing-arbitrary-locations
1411+ https://www.sphinx-doc.org/en/stable/markup/inline.html#cross-referencing-arbitrary-locations
14041412.. _sphinx.ext.extlinks:
14051413 https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html
1406−.. _intersphinx: http://www.sphinx-doc.org/en/stable/ext/intersphinx.html
1407−.. _sphinx config: http://www.sphinx-doc.org/en/stable/config.html
1408−.. _Sphinx's autodoc: http://www.sphinx-doc.org/en/stable/ext/autodoc.html
1414+.. _intersphinx: https://www.sphinx-doc.org/en/stable/ext/intersphinx.html
1415+.. _sphinx config: https://www.sphinx-doc.org/en/stable/config.html
1416+.. _Sphinx's autodoc: https://www.sphinx-doc.org/en/stable/ext/autodoc.html
14091417.. _Sphinx's Python domain:
1410− http://www.sphinx-doc.org/en/stable/domains.html#the-python-domain
1418+ https://www.sphinx-doc.org/en/stable/domains.html#the-python-domain
14111419.. _Sphinx's C domain:
1412− http://www.sphinx-doc.org/en/stable/domains.html#cross-referencing-c-constructs
1420+ https://www.sphinx-doc.org/en/stable/domains.html#cross-referencing-c-constructs
14131421.. _doctree:
1414− http://www.sphinx-doc.org/en/master/extdev/tutorial.html?highlight=doctree#build-phases
1422+ https://www.sphinx-doc.org/en/master/extdev/tutorial.html?highlight=doctree#build-phases
14151423.. _docutils: http://docutils.sourceforge.net/docs/index.html
14161424.. _docutils FAQ: http://docutils.sourceforge.net/FAQ.html
14171425.. _linuxdoc: https://return42.github.io/linuxdoc
@@ -1424,5 +1432,5 @@ The next example shows the difference of ``\tfrac`` (*textstyle*) and ``\dfrac``
14241432.. _ImageMagick: https://www.imagemagick.org
14251433
14261434.. _`Emacs Table Mode`: https://www.emacswiki.org/emacs/TableMode
1427−.. _`Online Tables Generator`: http://www.tablesgenerator.com/text_tables
1435+.. _`Online Tables Generator`: https://www.tablesgenerator.com/text_tables
14281436.. _`OASIS XML Exchange Table Model`: https://www.oasis-open.org/specs/tm9901.html
docs/dev/search_api.rst+2−2
@@ -6,7 +6,7 @@ Search API
66
77The search supports both ``GET`` and ``POST``.
88
9−Furthermore, two enpoints ``/`` and ``/search`` are available for querying.
9+Furthermore, two endpoints ``/`` and ``/search`` are available for querying.
1010
1111
1212``GET /``
@@ -76,7 +76,7 @@ Parameters
7676 supports safe search in the preferences page of an instance.
7777
7878``theme`` : default ``oscar``
79− [ ``oscar``, ``simple``, ``legacy``, ``pix-art``, ``courgette`` ]
79+ [ ``oscar``, ``simple`` ]
8080
8181 Theme of instance.
8282
docs/index.rst+1
@@ -30,6 +30,7 @@ anyone, you can set up your own, see :ref:`installation`.
3030 user/index
3131 admin/index
3232 dev/index
33+ searx_extra/index
3334 utils/index
3435 blog/index
3536
docs/searx_extra/index.rstnew file+14
@@ -0,0 +1,14 @@
1+.. _searx_extra:
2+
3+======================================================
4+Tooling box ``searx_extra`` for developers and users
5+======================================================
6+
7+In the folder :origin:`searx_extra/` we maintain some tools useful for
8+developers and users.
9+
10+.. toctree::
11+ :maxdepth: 2
12+ :caption: Contents
13+
14+ standalone_searx.py
docs/searx_extra/standalone_searx.py.rstnew file+9
@@ -0,0 +1,9 @@
1+
2+.. _standalone_searx.py:
3+
4+===================================
5+``searx_extra/standalone_searx.py``
6+===================================
7+
8+.. automodule:: searx_extra.standalone_searx
9+ :members:
docs/user/conf.pydeleted−19
@@ -1,19 +0,0 @@
1−# -*- coding: utf-8; mode: python -*-
2−"""Configuration for the Searx user handbook
3−"""
4−project = 'Searx User-HB'
5−version = release = VERSION_STRING
6−
7−# Grouping the document tree into LaTeX files. List of tuples
8−# (source start file, target name, title,
9−# author, documentclass [howto, manual, or own class]).
10−latex_documents = [
11− ('index' # startdocname
12− , 'searx-user-hb.tex' # targetname
13− , '' # take title from .rst
14− , author # author
15− , 'howto' # documentclass
16− , False # toctree_only
17− ),
18−]
19−
docs/user/own-instance.rst+3−3
@@ -56,9 +56,9 @@ results.
5656I see. What about private instances?
5757------------------------------------
5858
59−If users run their own instances, everything is in their control: the source
60−code, logging settings and private data. Unknown instance administrators do not
61−have to be trusted.
59+If users run their :ref:`own instances <installation>`, everything is in their
60+control: the source code, logging settings and private data. Unknown instance
61+administrators do not have to be trusted.
6262
6363Furthermore, as the default settings of their instance is editable, there is no
6464need to use cookies to tailor searx to their needs. So preferences will not be
docs/user/search_syntax.rst+24
@@ -40,3 +40,27 @@ Image search:
4040Custom language in wikipedia:
4141
4242- :search:`:hu !wp hackerspace <?q=%3Ahu%20%21wp%20hackerspace>`
43+
44+Multilingual Search
45+===================
46+
47+Searx does not support true multilingual search.
48+You have to use the language prefix in your search query when searching in a different language.
49+
50+But there is a workaround:
51+By adding a new search engine with a different language, Searx will search in your default and other language.
52+
53+Example configuration in settings.yml for a German and English speaker:
54+ .. code-block:: yaml
55+
56+ search:
57+ language : "de"
58+ ...
59+
60+ engines:
61+ - name : google english
62+ engine : google
63+ language : english
64+ ...
65+
66+When searching, the default google engine will return German results and "google english" will return English results.
docs/utils/index.rst+7−8
@@ -1,12 +1,11 @@
11.. _searx_utils:
22.. _toolboxing:
33
4−=======================
5−Tooling box ``utils/*``
6−=======================
4+===================
5+Admin's tooling box
6+===================
77
8−In the folder :origin:`utils/` we maintain some tools useful for admins and
9−developers.
8+In the folder :origin:`utils/` we maintain some tools useful for administrators.
109
1110.. toctree::
1211 :maxdepth: 2
@@ -40,14 +39,14 @@ Scripts to maintain services often dispose of common commands and environments.
4039 <lxc-searx.env>` use ::
4140
4241 sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/filtron.sh apache install
43−
42+
4443.. _toolboxing setup:
4544
4645Tooling box setup
4746=================
4847
49−The main setup is done in the :origin:`.config.sh` (read also :ref:`makefile
50−setup`).
48+The main setup is done in the :origin:`.config.sh` (read also :ref:`settings
49+global`).
5150
5251.. literalinclude:: ../../.config.sh
5352 :language: bash
docs/utils/lxc.sh.rst+3−3
@@ -119,15 +119,15 @@ of coffee).::
119119
120120To build (live) documentation inside a archlinux_ container::
121121
122− sudo -H ./utils/lxc.sh cmd searx-archlinux make docs-clean docs-live
122+ sudo -H ./utils/lxc.sh cmd searx-archlinux make docs.clean docs.live
123123 ...
124124 [I 200331 15:00:42 server:296] Serving on http://0.0.0.0:8080
125125
126126To get IP of the container and the port number *live docs* is listening::
127127
128− $ sudo ./utils/lxc.sh show suite | grep docs-live
128+ $ sudo ./utils/lxc.sh show suite | grep docs.live
129129 ...
130− [searx-archlinux] INFO: (eth0) docs-live: http://n.n.n.12:8080/
130+ [searx-archlinux] INFO: (eth0) docs.live: http://n.n.n.12:8080/
131131
132132
133133.. _lxc.sh help:
managenew file+504
@@ -0,0 +1,504 @@
1+#!/usr/bin/env bash
2+# -*- coding: utf-8; mode: sh indent-tabs-mode: nil -*-
3+# SPDX-License-Identifier: AGPL-3.0-or-later
4+# shellcheck disable=SC2031
5+
6+# shellcheck source=utils/lib.sh
7+source "$(dirname "${BASH_SOURCE[0]}")/utils/lib.sh"
8+# shellcheck source=utils/brand.env
9+source "${REPO_ROOT}/utils/brand.env"
10+source_dot_config
11+
12+# config
13+
14+PY_SETUP_EXTRAS='[test]'
15+NPM_PACKAGES="less@2.7 less-plugin-clean-css grunt-cli"
16+GECKODRIVER_VERSION="v0.30.0"
17+# SPHINXOPTS=
18+
19+# These py files are linted by test.pylint(), all other files are linted by
20+# test.pep8()
21+PYLINT_FILES=(
22+ searx/preferences.py
23+ searx/testing.py
24+ searx/engines/gigablast.py
25+ searx/engines/deviantart.py
26+ searx/engines/digg.py
27+ searx/engines/google.py
28+ searx/engines/google_news.py
29+ searx/engines/google_videos.py
30+ searx/engines/google_images.py
31+ searx/engines/mediathekviewweb.py
32+ searx/engines/meilisearch.py
33+ searx/engines/solidtorrents.py
34+ searx/engines/solr.py
35+ searx/engines/sqlite.py
36+ searx/engines/springer.py
37+ searx/engines/google_scholar.py
38+ searx/engines/yahoo_news.py
39+ searx/engines/apkmirror.py
40+ searx/engines/core.py
41+ searx_extra/update/update_external_bangs.py
42+)
43+
44+PYLINT_SEARX_DISABLE_OPTION="\
45+I,C,R,\
46+W0105,W0212,W0511,W0603,W0613,W0621,W0702,W0703,W1401,\
47+E1136"
48+PYLINT_ADDITIONAL_BUILTINS_FOR_ENGINES="supported_languages,language_aliases"
49+PYLINT_OPTIONS="-m pylint -j 0 --rcfile .pylintrc"
50+
51+help() {
52+ cat <<EOF
53+buildenv
54+ rebuild ./utils/brand.env
55+babel.compile
56+ pybabel compile ./searx/translations
57+data.*
58+ all : update searx/languages.py and ./data/*
59+ languages : update searx/data/engines_languages.json & searx/languages.py
60+ useragents: update searx/data/useragents.json with the most recent versions of Firefox.
61+docs.*
62+ html : build HTML documentation
63+ live : autobuild HTML documentation while editing
64+ gh-pages : deploy on gh-pages branch
65+ prebuild : build reST include files (./${DOCS_BUILD}/includes)
66+ clean : clean documentation build
67+docker
68+ build : build docker image
69+ push : build and push docker image
70+gecko.driver
71+ download & install geckodriver if not already installed (required for
72+ robot_tests)
73+node.*
74+ env : download & install npm dependencies locally
75+ clean : drop npm installations
76+py.*
77+ build : Build python packages at ./${PYDIST}
78+ clean : delete virtualenv and intermediate py files
79+pyenv.* :
80+ install : developer install of searx into virtualenv
81+ uninstall : uninstall developer installation
82+ cmd ... : run command ... in virtualenv
83+ OK : test if virtualenv is OK
84+pypi.upload:
85+ Upload python packages to PyPi (to test use pypi.upload.test)
86+test.* :
87+ pylint : lint PYLINT_FILES, searx/engines, searx & tests
88+ pep8 : pycodestyle (pep8) for all files except PYLINT_FILES
89+ unit : run unit tests
90+ coverage : run unit tests with coverage
91+ robot : run robot test
92+ clean : clean intermediate test stuff
93+themes.* :
94+ all : build all themes
95+ oscar : build oscar theme
96+ simple : build simple theme
97+EOF
98+}
99+
100+
101+if [ "$VERBOSE" = "1" ]; then
102+ SPHINX_VERBOSE="-v"
103+ PYLINT_VERBOSE="-v"
104+fi
105+
106+# needed by sphinx-docs
107+export DOCS_BUILD
108+
109+buildenv() {
110+ SEARX_DEBUG=1 pyenv.cmd python utils/build_env.py 2>&1 \
111+ | prefix_stdout "${_Blue}BUILDENV${_creset} "
112+ return "${PIPESTATUS[0]}"
113+}
114+
115+babel.compile() {
116+ build_msg BABEL compile
117+ pyenv.cmd pybabel compile -d "${REPO_ROOT}/searx/translations"
118+ dump_return $?
119+}
120+
121+
122+data.all() {
123+ data.languages
124+ data.useragents
125+ build_msg DATA "update searx/data/ahmia_blacklist.txt"
126+ pyenv.cmd python searx_extra/update/update_ahmia_blacklist.py
127+ build_msg DATA "update searx/data/wikidata_units.json"
128+ pyenv.cmd python searx_extra/update/update_wikidata_units.py
129+ build_msg DATA "update searx/data/currencies.json"
130+ pyenv.cmd python searx_extra/update/update_currencies.py
131+}
132+
133+
134+data.languages() {
135+ ( set -e
136+ build_msg ENGINES "fetch languages .."
137+ pyenv.cmd python searx_extra/update/update_languages.py
138+ build_msg ENGINES "update update searx/languages.py"
139+ build_msg DATA "update searx/data/engines_languages.json"
140+ )
141+ dump_return $?
142+}
143+
144+data.useragents() {
145+ build_msg DATA "update searx/data/useragents.json"
146+ pyenv.cmd python searx_extra/update/update_firefox_version.py
147+ dump_return $?
148+}
149+
150+docs.prebuild() {
151+ build_msg DOCS "build ${DOCS_BUILD}/includes"
152+ (
153+ set -e
154+ [ "$VERBOSE" = "1" ] && set -x
155+ mkdir -p "${DOCS_BUILD}/includes"
156+ ./utils/searx.sh doc | cat > "${DOCS_BUILD}/includes/searx.rst"
157+ ./utils/filtron.sh doc | cat > "${DOCS_BUILD}/includes/filtron.rst"
158+ ./utils/morty.sh doc | cat > "${DOCS_BUILD}/includes/morty.rst"
159+ )
160+ dump_return $?
161+}
162+
163+docker.push() {
164+ docker.build push
165+}
166+
167+# shellcheck disable=SC2119
168+docker.build() {
169+ pyenv.install
170+
171+ build_msg DOCKER build
172+ # run installation in a subprocess and activate pyenv
173+
174+ # See https://www.shellcheck.net/wiki/SC1001 and others ..
175+ # shellcheck disable=SC2031,SC2230,SC2002,SC2236,SC2143,SC1001
176+ ( set -e
177+ # shellcheck source=/dev/null
178+ source "${PY_ENV_BIN}/activate"
179+
180+ # Check if it is a git repository
181+ if [ ! -d .git ]; then
182+ die 1 "This is not Git repository"
183+ fi
184+ if [ ! -x "$(which git)" ]; then
185+ die 1 "git is not installed"
186+ fi
187+
188+ if ! git remote get-url origin 2> /dev/null; then
189+ die 1 "there is no remote origin"
190+ fi
191+
192+ # This is a git repository
193+
194+ # "git describe" to get the Docker version (for example : v0.15.0-89-g0585788e)
195+ # awk to remove the "v" and the "g"
196+ SEARX_GIT_VERSION=$(git describe --match "v[0-9]*\.[0-9]*\.[0-9]*" HEAD 2>/dev/null | awk -F'-' '{OFS="-"; $1=substr($1, 2); if ($3) { $3=substr($3, 2); } print}')
197+
198+ # add the suffix "-dirty" if the repository has uncommited change
199+ # /!\ HACK for searx/searx: ignore utils/brand.env
200+ git update-index -q --refresh
201+ if [ ! -z "$(git diff-index --name-only HEAD -- | grep -v 'utils/brand.env')" ]; then
202+ SEARX_GIT_VERSION="${SEARX_GIT_VERSION}-dirty"
203+ fi
204+
205+ # Get the last git commit id, will be added to the Searx version (see Dockerfile)
206+ VERSION_GITCOMMIT=$(echo "$SEARX_GIT_VERSION" | cut -d- -f2-4)
207+ build_msg DOCKER "Last commit : $VERSION_GITCOMMIT"
208+
209+ # Check consistency between the git tag and the searx/version.py file
210+ # /! HACK : parse Python file with bash /!
211+ # otherwise it is not possible build the docker image without all Python
212+ # dependencies ( version.py loads __init__.py )
213+ # SEARX_PYTHON_VERSION=$(python3 -c "import six; import searx.version; six.print_(searx.version.VERSION_STRING)")
214+ SEARX_PYTHON_VERSION=$(cat searx/version.py | grep "\(VERSION_MAJOR\|VERSION_MINOR\|VERSION_BUILD\) =" | cut -d\= -f2 | sed -e 's/^[[:space:]]*//' | paste -sd "." -)
215+ if [ "$(echo "$SEARX_GIT_VERSION" | cut -d- -f1)" != "$SEARX_PYTHON_VERSION" ]; then
216+ err_msg "git tag: $SEARX_GIT_VERSION"
217+ err_msg "searx/version.py: $SEARX_PYTHON_VERSION"
218+ die 1 "Inconsistency between the last git tag and the searx/version.py file"
219+ fi
220+
221+ # define the docker image name
222+ GITHUB_USER=$(echo "${GIT_URL}" | sed 's/.*github\.com\/\([^\/]*\).*/\1/')
223+ SEARX_IMAGE_NAME="${SEARX_IMAGE_NAME:-${GITHUB_USER:-searx}/searx}"
224+
225+ # build Docker image
226+ build_msg DOCKER "Building image ${SEARX_IMAGE_NAME}:${SEARX_GIT_VERSION}"
227+ docker build \
228+ --build-arg GIT_URL="${GIT_URL}" \
229+ --build-arg SEARX_GIT_VERSION="${SEARX_GIT_VERSION}" \
230+ --build-arg VERSION_GITCOMMIT="${VERSION_GITCOMMIT}" \
231+ --build-arg LABEL_DATE="$(date -u +"%Y-%m-%dT%H:%M:%SZ")" \
232+ --build-arg LABEL_VCS_REF="$(git rev-parse HEAD)" \
233+ --build-arg LABEL_VCS_URL="${GIT_URL}" \
234+ --build-arg TIMESTAMP_SETTINGS="$(git log -1 --format="%cd" --date=unix -- searx/settings.yml)" \
235+ --build-arg TIMESTAMP_UWSGI="$(git log -1 --format="%cd" --date=unix -- dockerfiles/uwsgi.ini)" \
236+ -t "${SEARX_IMAGE_NAME}:latest" -t "${SEARX_IMAGE_NAME}:${SEARX_GIT_VERSION}" .
237+
238+ if [ "$1" = "push" ]; then
239+ docker push "${SEARX_IMAGE_NAME}:latest"
240+ docker push "${SEARX_IMAGE_NAME}:${SEARX_GIT_VERSION}"
241+ fi
242+ )
243+ dump_return $?
244+}
245+
246+# shellcheck disable=SC2119
247+gecko.driver() {
248+ pyenv.install
249+
250+ build_msg INSTALL "gecko.driver"
251+ # run installation in a subprocess and activate pyenv
252+ ( set -e
253+ # shellcheck source=/dev/null
254+ source "${PY_ENV_BIN}/activate"
255+
256+ # TODO : check the current geckodriver version
257+ geckodriver -V > /dev/null 2>&1 || NOTFOUND=1
258+ set +e
259+ if [ -z "$NOTFOUND" ]; then
260+ build_msg INSTALL "geckodriver already installed"
261+ return
262+ fi
263+ PLATFORM="$(python3 -c 'import platform; print(platform.system().lower(), platform.architecture()[0])')"
264+ case "$PLATFORM" in
265+ "linux 32bit" | "linux2 32bit") ARCH="linux32";;
266+ "linux 64bit" | "linux2 64bit") ARCH="linux64";;
267+ "windows 32 bit") ARCH="win32";;
268+ "windows 64 bit") ARCH="win64";;
269+ "mac 64bit") ARCH="macos";;
270+ esac
271+ GECKODRIVER_URL="https://github.com/mozilla/geckodriver/releases/download/$GECKODRIVER_VERSION/geckodriver-$GECKODRIVER_VERSION-$ARCH.tar.gz";
272+
273+ build_msg GECKO "Installing ${PY_ENV_BIN}/geckodriver from $GECKODRIVER_URL"
274+
275+ FILE="$(mktemp)"
276+ wget -qO "$FILE" -- "$GECKODRIVER_URL" && tar xz -C "${PY_ENV_BIN}" -f "$FILE" geckodriver
277+ rm -- "$FILE"
278+ chmod 755 -- "${PY_ENV_BIN}/geckodriver"
279+ )
280+ dump_return $?
281+}
282+
283+node.env() {
284+ local err=0
285+ pushd "${REPO_ROOT}" &> /dev/null
286+ # shellcheck disable=SC2230
287+ which npm &> /dev/null || die 1 'node.env - npm is not found!'
288+
289+ ( set -e
290+ # shellcheck disable=SC2030
291+ PATH="$(npm bin):$PATH"
292+ export PATH
293+
294+ build_msg INSTALL "npm install $NPM_PACKAGES"
295+ # shellcheck disable=SC2086
296+ npm install $NPM_PACKAGES
297+
298+ cd "${REPO_ROOT}/searx/static/themes/oscar"
299+ build_msg INSTALL "($(pwd)) npm install"
300+ npm install
301+
302+ build_msg INSTALL "($(pwd)) npm install"
303+ cd "${REPO_ROOT}/searx/static/themes/simple"
304+ npm install
305+ )
306+ err=$?
307+ popd &> /dev/null
308+ dump_return "$err"
309+}
310+
311+node.clean() {
312+
313+ build_msg CLEAN "locally installed npm dependencies"
314+ rm -rf \
315+ ./node_modules \
316+ ./package-lock.json \
317+ ./searx/static/themes/oscar/package-lock.json \
318+ ./searx/static/themes/oscar/node_modules \
319+ ./searx/static/themes/simple/package-lock.json \
320+ ./searx/static/themes/simple/node_modules
321+ dump_return $?
322+}
323+
324+py.build() {
325+ build_msg BUILD "[pylint] python package ${PYDIST}"
326+ pyenv.cmd python setup.py \
327+ sdist -d "${PYDIST}" \
328+ bdist_wheel --bdist-dir "${PYBUILD}" -d "${PYDIST}"
329+}
330+
331+py.clean() {
332+ build_msg CLEAN pyenv
333+ ( set -e
334+ pyenv.drop
335+ [ "$VERBOSE" = "1" ] && set -x
336+ rm -rf "${PYDIST}" "${PYBUILD}" "${PY_ENV}" ./.tox ./*.egg-info
337+ find . -name '*.pyc' -exec rm -f {} +
338+ find . -name '*.pyo' -exec rm -f {} +
339+ find . -name __pycache__ -exec rm -rf {} +
340+ )
341+}
342+
343+pyenv.check() {
344+ cat <<EOF
345+import yaml
346+print('import yaml --> OK')
347+EOF
348+}
349+
350+pyenv.install() {
351+
352+ if ! pyenv.OK; then
353+ py.clean > /dev/null
354+ fi
355+ if pyenv.install.OK > /dev/null; then
356+ return 0
357+ fi
358+ ( set -e
359+ pyenv
360+ build_msg PYENV "[install] pip install -e 'searx${PY_SETUP_EXTRAS}'"
361+ "${PY_ENV_BIN}/python" -m pip install -e ".${PY_SETUP_EXTRAS}"
362+ buildenv
363+ ) || die 42 "error while build & install pyenv (${PY_ENV_BIN})"
364+}
365+
366+pyenv.uninstall() {
367+ build_msg PYENV "[pyenv.uninstall] uninstall packages: ${PYOBJECTS}"
368+ pyenv.cmd python setup.py develop --uninstall 2>&1 \
369+ | prefix_stdout "${_Blue}PYENV ${_creset}[pyenv.uninstall] "
370+
371+}
372+
373+pypi.upload() {
374+ py.clean
375+ py.build
376+ # https://github.com/pypa/twine
377+ pyenv.cmd twine upload "${PYDIST}"/*
378+}
379+
380+pypi.upload.test() {
381+ py.clean
382+ py.build
383+ pyenv.cmd twine upload -r testpypi "${PYDIST}"/*
384+}
385+
386+test.pylint() {
387+ # shellcheck disable=SC2086
388+ ( set -e
389+ build_msg TEST "[pylint] \$PYLINT_FILES"
390+ pyenv.cmd python ${PYLINT_OPTIONS} ${PYLINT_VERBOSE} \
391+ "${PYLINT_FILES[@]}"
392+
393+ build_msg TEST "[pylint] searx/engines"
394+ pyenv.cmd python ${PYLINT_OPTIONS} ${PYLINT_VERBOSE} \
395+ --disable="${PYLINT_SEARX_DISABLE_OPTION}" \
396+ --additional-builtins="${PYLINT_ADDITIONAL_BUILTINS_FOR_ENGINES}" \
397+ searx/engines
398+
399+ build_msg TEST "[pylint] searx tests"
400+ pyenv.cmd python ${PYLINT_OPTIONS} ${PYLINT_VERBOSE} \
401+ --disable="${PYLINT_SEARX_DISABLE_OPTION}" \
402+ --ignore=searx/engines \
403+ tests
404+ )
405+ dump_return $?
406+}
407+
408+test.pep8() {
409+ build_msg TEST 'pycodestyle (formerly pep8)'
410+ local _exclude=""
411+ printf -v _exclude '%s, ' "${PYLINT_FILES[@]}"
412+ pyenv.cmd pycodestyle \
413+ --exclude="searx/static, searx/languages.py, $_exclude " \
414+ --max-line-length=120 \
415+ --ignore "E117,E252,E402,E722,E741,W503,W504,W605" \
416+ searx tests
417+ dump_return $?
418+}
419+
420+test.unit() {
421+ build_msg TEST 'tests/unit'
422+ pyenv.cmd python -m nose2 -s tests/unit
423+ dump_return $?
424+}
425+
426+test.coverage() {
427+ build_msg TEST 'unit test coverage'
428+ ( set -e
429+ pyenv.cmd python -m nose2 -C --log-capture --with-coverage --coverage searx -s tests/unit
430+ pyenv.cmd coverage report
431+ pyenv.cmd coverage html
432+ )
433+ dump_return $?
434+}
435+
436+test.robot() {
437+ build_msg TEST 'robot'
438+ gecko.driver
439+ PYTHONPATH=. pyenv.cmd python searx/testing.py robot
440+ dump_return $?
441+}
442+
443+test.clean() {
444+ build_msg CLEAN "test stuff"
445+ rm -rf geckodriver.log .coverage coverage/
446+ dump_return $?
447+}
448+
449+themes.all() {
450+ ( set -e
451+ node.env
452+ themes.oscar
453+ themes.simple
454+ )
455+ dump_return $?
456+}
457+
458+themes.oscar() {
459+ local gruntfile=searx/static/themes/oscar/gruntfile.js
460+ build_msg GRUNT "${gruntfile}"
461+ PATH="$(npm bin):$PATH" grunt --gruntfile "${gruntfile}"
462+ dump_return $?
463+}
464+
465+themes.simple() {
466+ local gruntfile=searx/static/themes/simple/gruntfile.js
467+ build_msg GRUNT "${gruntfile}"
468+ PATH="$(npm bin):$PATH" grunt --gruntfile "${gruntfile}"
469+ dump_return $?
470+}
471+
472+# shellcheck disable=SC2119
473+main() {
474+
475+ local _type
476+ local cmd="$1"; shift
477+
478+ if [ "$cmd" == "" ]; then
479+ help
480+ err_msg "missing command"
481+ return 42
482+ fi
483+
484+ case "$cmd" in
485+ --getenv) var="$1"; echo "${!var}";;
486+ --help) help;;
487+ --*)
488+ help
489+ err_msg "unknown option $cmd"
490+ return 42
491+ ;;
492+ *)
493+ _type="$(type -t "$cmd")"
494+ if [ "$_type" != 'function' ]; then
495+ err_msg "unknown command $1 / use --help"
496+ return 42
497+ else
498+ "$cmd" "$@"
499+ fi
500+ ;;
501+ esac
502+}
503+
504+main "$@"
manage.shdeleted−205
@@ -1,205 +0,0 @@
1−#!/bin/sh
2−
3−export LANG=C
4−
5−BASE_DIR="$(dirname -- "`readlink -f -- "$0"`")"
6−
7−cd -- "$BASE_DIR"
8−set -e
9−
10−# subshell
11−PYTHONPATH="$BASE_DIR"
12−SEARX_DIR="$BASE_DIR/searx"
13−ACTION="$1"
14−
15−. "${BASE_DIR}/utils/brand.env"
16−
17−#
18−# Python
19−#
20−
21−update_packages() {
22− pip install --upgrade pip
23− pip install --upgrade setuptools
24− pip install -Ur "$BASE_DIR/requirements.txt"
25−}
26−
27−update_dev_packages() {
28− update_packages
29− pip install -Ur "$BASE_DIR/requirements-dev.txt"
30−}
31−
32−install_geckodriver() {
33− echo '[!] Checking geckodriver'
34− # TODO : check the current geckodriver version
35− set -e
36− geckodriver -V > /dev/null 2>&1 || NOTFOUND=1
37− set +e
38− if [ -z "$NOTFOUND" ]; then
39− return
40− fi
41− GECKODRIVER_VERSION="v0.24.0"
42− PLATFORM="`python3 -c "import platform; print(platform.system().lower(), platform.architecture()[0])"`"
43− case "$PLATFORM" in
44− "linux 32bit" | "linux2 32bit") ARCH="linux32";;
45− "linux 64bit" | "linux2 64bit") ARCH="linux64";;
46− "windows 32 bit") ARCH="win32";;
47− "windows 64 bit") ARCH="win64";;
48− "mac 64bit") ARCH="macos";;
49− esac
50− GECKODRIVER_URL="https://github.com/mozilla/geckodriver/releases/download/$GECKODRIVER_VERSION/geckodriver-$GECKODRIVER_VERSION-$ARCH.tar.gz";
51−
52− if [ -z "$1" ]; then
53− if [ -z "$VIRTUAL_ENV" ]; then
54− printf "geckodriver can't be installed because VIRTUAL_ENV is not set, you should download it from\n %s" "$GECKODRIVER_URL"
55− exit
56− else
57− GECKODRIVER_DIR="$VIRTUAL_ENV/bin"
58− fi
59− else
60− GECKODRIVER_DIR="$1"
61− mkdir -p -- "$GECKODRIVER_DIR"
62− fi
63−
64− printf "Installing %s/geckodriver from\n %s" "$GECKODRIVER_DIR" "$GECKODRIVER_URL"
65−
66− FILE="`mktemp`"
67− wget -qO "$FILE" -- "$GECKODRIVER_URL" && tar xz -C "$GECKODRIVER_DIR" -f "$FILE" geckodriver
68− rm -- "$FILE"
69− chmod 777 -- "$GECKODRIVER_DIR/geckodriver"
70−}
71−
72−locales() {
73− pybabel compile -d "$SEARX_DIR/translations"
74−}
75−
76−
77−#
78−# Web
79−#
80−
81−npm_path_setup() {
82− which npm || (printf 'Error: npm is not found\n'; exit 1)
83− export PATH="$(npm bin)":$PATH
84−}
85−
86−npm_packages() {
87− npm_path_setup
88−
89− echo '[!] install NPM packages'
90− cd -- "$BASE_DIR"
91− npm install less@2.7 less-plugin-clean-css grunt-cli
92−
93− echo '[!] install NPM packages for oscar theme'
94− cd -- "$BASE_DIR/searx/static/themes/oscar"
95− npm install
96−
97− echo '[!] install NPM packages for simple theme'
98− cd -- "$BASE_DIR/searx/static/themes/simple"
99− npm install
100−}
101−
102−docker_build() {
103− # Check if it is a git repository
104− if [ ! -d .git ]; then
105− echo "This is not Git repository"
106− exit 1
107− fi
108−
109− if [ ! -x "$(which git)" ]; then
110− echo "git is not installed"
111− exit 1
112− fi
113−
114− if [ ! git remote get-url origin 2> /dev/null ]; then
115− echo "there is no remote origin"
116− exit 1
117− fi
118−
119− # This is a git repository
120−
121− # "git describe" to get the Docker version (for example : v0.15.0-89-g0585788e)
122− # awk to remove the "v" and the "g"
123− SEARX_GIT_VERSION=$(git describe --match "v[0-9]*\.[0-9]*\.[0-9]*" HEAD 2>/dev/null | awk -F'-' '{OFS="-"; $1=substr($1, 2); if ($3) { $3=substr($3, 2); } print}')
124−
125− # add the suffix "-dirty" if the repository has uncommited change
126− # /!\ HACK for searx/searx: ignore searx/brand.py and utils/brand.env
127− git update-index -q --refresh
128− if [ ! -z "$(git diff-index --name-only HEAD -- | grep -v 'searx/brand.py' | grep -v 'utils/brand.env')" ]; then
129− SEARX_GIT_VERSION="${SEARX_GIT_VERSION}-dirty"
130− fi
131−
132− # Get the last git commit id, will be added to the Searx version (see Dockerfile)
133− VERSION_GITCOMMIT=$(echo $SEARX_GIT_VERSION | cut -d- -f2-4)
134− echo "Last commit : $VERSION_GITCOMMIT"
135−
136− # Check consistency between the git tag and the searx/version.py file
137− # /!\ HACK : parse Python file with bash /!\
138− # otherwise it is not possible build the docker image without all Python dependencies ( version.py loads __init__.py )
139− # SEARX_PYTHON_VERSION=$(python3 -c "import six; import searx.version; six.print_(searx.version.VERSION_STRING)")
140− SEARX_PYTHON_VERSION=$(cat searx/version.py | grep "\(VERSION_MAJOR\|VERSION_MINOR\|VERSION_BUILD\) =" | cut -d\= -f2 | sed -e 's/^[[:space:]]*//' | paste -sd "." -)
141− if [ $(echo "$SEARX_GIT_VERSION" | cut -d- -f1) != "$SEARX_PYTHON_VERSION" ]; then
142− echo "Inconsistency between the last git tag and the searx/version.py file"
143− echo "git tag: $SEARX_GIT_VERSION"
144− echo "searx/version.py: $SEARX_PYTHON_VERSION"
145− exit 1
146− fi
147−
148− # define the docker image name
149− GITHUB_USER=$(echo "${GIT_URL}" | sed 's/.*github\.com\/\([^\/]*\).*/\1/')
150− SEARX_IMAGE_NAME="${GITHUB_USER:-searx}/searx"
151−
152− # build Docker image
153− echo "Building image ${SEARX_IMAGE_NAME}:${SEARX_GIT_VERSION}"
154− sudo docker build \
155− --build-arg GIT_URL="${GIT_URL}" \
156− --build-arg SEARX_GIT_VERSION="${SEARX_GIT_VERSION}" \
157− --build-arg VERSION_GITCOMMIT="${VERSION_GITCOMMIT}" \
158− --build-arg LABEL_DATE=$(date -u +"%Y-%m-%dT%H:%M:%SZ") \
159− --build-arg LABEL_VCS_REF=$(git rev-parse HEAD) \
160− --build-arg LABEL_VCS_URL="${GIT_URL}" \
161− --build-arg TIMESTAMP_SETTINGS=$(git log -1 --format="%cd" --date=unix -- searx/settings.yml) \
162− --build-arg TIMESTAMP_UWSGI=$(git log -1 --format="%cd" --date=unix -- dockerfiles/uwsgi.ini) \
163− -t ${SEARX_IMAGE_NAME}:latest -t ${SEARX_IMAGE_NAME}:${SEARX_GIT_VERSION} .
164−
165− if [ "$1" = "push" ]; then
166− sudo docker push ${SEARX_IMAGE_NAME}:latest
167− sudo docker push ${SEARX_IMAGE_NAME}:${SEARX_GIT_VERSION}
168− fi
169−}
170−
171−#
172−# Help
173−#
174−
175−help() {
176− [ -z "$1" ] || printf 'Error: %s\n' "$1"
177− echo "Searx manage.sh help
178−
179−Commands
180−========
181− help - This text
182−
183− Build requirements
184− ------------------
185− update_packages - Check & update production dependency changes
186− update_dev_packages - Check & update development and production dependency changes
187− install_geckodriver - Download & install geckodriver if not already installed (required for robot_tests)
188− npm_packages - Download & install npm dependencies
189−
190− Build
191− -----
192− locales - Compile locales
193−
194−Environment:
195− GIT_URL: ${GIT_URL}
196− ISSUE_URL: ${ISSUE_URL}
197− SEARX_URL: ${SEARX_URL}
198− DOCS_URL: ${DOCS_URL}
199− PUBLIC_INSTANCES: ${PUBLIC_INSTANCES}
200−"
201−}
202−
203−[ "$(command -V "$ACTION" | grep ' function$')" = "" ] \
204− && help "action not found" \
205− || "$ACTION" "$2"
requirements-dev.txt+19−23
@@ -1,24 +1,20 @@
1−mock==2.0.0
2−nose2[coverage_plugin]==0.9.2
1+mock==4.0.3
2+nose2[coverage_plugin]==0.10.0
33cov-core==1.15.0
4−pep8==1.7.0
5−pylint==2.4.4
6−plone.testing==5.0.0
7−splinter==0.11.0
8−transifex-client==0.12.2
9−unittest2==1.1.0
10−zope.testrunner==4.5.1
11−selenium==3.141.0
12−twine==3.2.0; python_version >= "3.6"
13−twine==1.15.0; python_version < "3.6"
14−Pallets-Sphinx-Themes==1.2.3
15−Sphinx==3.2.1; python_version >= '3.6'
16−Sphinx==3.0.1; python_version < '3.6'
17−sphinx-issues==1.2.0
18−sphinx-jinja==1.1.1
19−sphinx-tabs==1.3.0; python_version >= '3.6'
20−sphinx-tabs==1.1.13; python_version < '3.6'
21−sphinxcontrib-programoutput==0.16
22−sphinx-autobuild==2020.9.1; python_version >= '3.6'
23−sphinx-autobuild==0.7.1; python_version < '3.6'
24−linuxdoc @ git+http://github.com/return42/linuxdoc.git@70673dcf69e705e08d81f53794895dc15c4920b3#egg=linuxdoc
4+pycodestyle==2.8.0
5+pylint==2.12.2
6+splinter==0.17.0
7+transifex-client==0.14.3; python_version < '3.10'
8+transifex-client==0.12.1; python_version == '3.10'
9+selenium==4.1.0
10+twine==3.7.1
11+Pallets-Sphinx-Themes==2.0.2
12+docutils==0.16
13+Sphinx==4.4.0
14+sphinx-issues==3.0.1
15+sphinx-jinja==1.4.0
16+sphinx-tabs==3.2.0
17+sphinxcontrib-programoutput==0.17
18+sphinx-autobuild==2021.3.14
19+linuxdoc==20211220
20+aiounittest==1.4.1
requirements.txt+16−12
@@ -1,12 +1,16 @@
1−certifi==2020.6.20
2−babel==2.7.0
3−flask-babel==1.0.0
4−flask==1.1.2
5−idna==2.10
6−jinja2==2.11.1
7−lxml==4.5.0
8−pygments==2.1.3
9−pyopenssl==19.1.0
10−python-dateutil==2.8.0
11−pyyaml==5.3.1
12−requests[socks]==2.24.0
1+certifi==2021.10.8
2+babel==2.9.1
3+flask-babel==2.0.0
4+flask==2.0.2
5+jinja2==3.0.3
6+lxml==4.7.1
7+pygments==2.8.0
8+python-dateutil==2.8.2
9+pyyaml==6.0
10+httpx[http2]==0.21.3
11+Brotli==1.0.9
12+uvloop==0.16.0; python_version >= '3.7'
13+uvloop==0.14.0; python_version < '3.7'
14+httpx-socks[asyncio]==0.7.2
15+langdetect==1.0.9
16+setproctitle==1.2.2
searx/__init__.py+48−31
@@ -16,39 +16,15 @@ along with searx. If not, see < http://www.gnu.org/licenses/ >.
1616'''
1717
1818import logging
19+import searx.settings_loader
1920from os import environ
2021from os.path import realpath, dirname, join, abspath, isfile
21−from io import open
22−from yaml import safe_load
2322
2423
2524searx_dir = abspath(dirname(__file__))
2625engine_dir = dirname(realpath(__file__))
2726static_path = abspath(join(dirname(__file__), 'static'))
28−
29−
30−def check_settings_yml(file_name):
31− if isfile(file_name):
32− return file_name
33− else:
34− return None
35−
36−
37−# find location of settings.yml
38−if 'SEARX_SETTINGS_PATH' in environ:
39− # if possible set path to settings using the
40− # enviroment variable SEARX_SETTINGS_PATH
41− settings_path = check_settings_yml(environ['SEARX_SETTINGS_PATH'])
42−else:
43− # if not, get it from searx code base or last solution from /etc/searx
44− settings_path = check_settings_yml(join(searx_dir, 'settings.yml')) or check_settings_yml('/etc/searx/settings.yml')
45−
46−if not settings_path:
47− raise Exception('settings.yml not found')
48−
49−# load settings
50−with open(settings_path, 'r', encoding='utf-8') as settings_yaml:
51− settings = safe_load(settings_yaml)
27+settings, settings_load_message = searx.settings_loader.load_settings()
5228
5329if settings['ui']['static_path']:
5430 static_path = settings['ui']['static_path']
@@ -58,7 +34,6 @@ enable debug if
5834the environnement variable SEARX_DEBUG is 1 or true
5935(whatever the value in settings.yml)
6036or general.debug=True in settings.yml
61−
6237disable debug if
6338the environnement variable SEARX_DEBUG is 0 or false
6439(whatever the value in settings.yml)
@@ -78,7 +53,7 @@ else:
7853 logging.basicConfig(level=logging.WARNING)
7954
8055logger = logging.getLogger('searx')
81−logger.debug('read configuration from %s', settings_path)
56+logger.info(settings_load_message)
8257logger.info('Initialisation done')
8358
8459if 'SEARX_SECRET' in environ:
@@ -86,6 +61,48 @@ if 'SEARX_SECRET' in environ:
8661if 'SEARX_BIND_ADDRESS' in environ:
8762 settings['server']['bind_address'] = environ['SEARX_BIND_ADDRESS']
8863
89−if not searx_debug and settings['server']['secret_key'] == 'ultrasecretkey':
90− logger.error('server.secret_key is not changed. Please use something else instead of ultrasecretkey.')
91− exit(1)
64+
65+class _brand_namespace:
66+
67+ @classmethod
68+ def get_val(cls, group, name, default=''):
69+ return settings.get(group, {}).get(name) or default
70+
71+ @property
72+ def SEARX_URL(self):
73+ return self.get_val('server', 'base_url')
74+
75+ @property
76+ def CONTACT_URL(self):
77+ return self.get_val('general', 'contact_url')
78+
79+ @property
80+ def GIT_URL(self):
81+ return self.get_val('brand', 'git_url')
82+
83+ @property
84+ def GIT_BRANCH(self):
85+ return self.get_val('brand', 'git_branch')
86+
87+ @property
88+ def ISSUE_URL(self):
89+ return self.get_val('brand', 'issue_url')
90+
91+ @property
92+ def DOCS_URL(self):
93+ return self.get_val('brand', 'docs_url')
94+
95+ @property
96+ def PUBLIC_INSTANCES(self):
97+ return self.get_val('brand', 'public_instances')
98+
99+ @property
100+ def WIKI_URL(self):
101+ return self.get_val('brand', 'wiki_url')
102+
103+ @property
104+ def TWITTER_URL(self):
105+ return self.get_val('brand', 'twitter_url')
106+
107+
108+brand = _brand_namespace()
searx/answerers/__init__.py+1−1
@@ -32,7 +32,7 @@ def ask(query):
3232 results = []
3333 query_parts = list(filter(None, query.query.split()))
3434
35− if query_parts[0] not in answerers_by_keywords:
35+ if not query_parts or query_parts[0] not in answerers_by_keywords:
3636 return results
3737
3838 for answerer in answerers_by_keywords[query_parts[0]]:
searx/autocomplete.py+38−87
@@ -20,100 +20,41 @@ from lxml import etree
2020from json import loads
2121from urllib.parse import urlencode
2222
23+from httpx import HTTPError
24+
25+
2326from searx import settings
24−from searx.languages import language_codes
25−from searx.engines import (
26− categories, engines, engine_shortcuts
27−)
28−from searx.poolrequests import get as http_get
27+from searx.data import ENGINES_LANGUAGES
28+from searx.network import get as http_get
29+from searx.exceptions import SearxEngineResponseException
2930
3031
3132def get(*args, **kwargs):
3233 if 'timeout' not in kwargs:
3334 kwargs['timeout'] = settings['outgoing']['request_timeout']
34−
35+ kwargs['raise_for_httperror'] = True
3536 return http_get(*args, **kwargs)
3637
3738
38−def searx_bang(full_query):
39− '''check if the searchQuery contain a bang, and create fitting autocompleter results'''
40− # check if there is a query which can be parsed
41− if len(full_query.getQuery()) == 0:
42− return []
39+def brave(query, lang):
40+ # brave search autocompleter
41+ url = 'https://search.brave.com/api/suggest?{query}'
42+
43+ resp = get(url.format(query=urlencode({'q': query})))
4344
4445 results = []
4546
46− # check if current query stats with !bang
47− first_char = full_query.getQuery()[0]
48− if first_char == '!' or first_char == '?':
49− if len(full_query.getQuery()) == 1:
50− # show some example queries
51− # TODO, check if engine is not avaliable
52− results.append(first_char + "images")
53− results.append(first_char + "wikipedia")
54− results.append(first_char + "osm")
55− else:
56− engine_query = full_query.getQuery()[1:]
57−
58− # check if query starts with categorie name
59− for categorie in categories:
60− if categorie.startswith(engine_query):
61− results.append(first_char + '{categorie}'.format(categorie=categorie))
62−
63− # check if query starts with engine name
64− for engine in engines:
65− if engine.startswith(engine_query.replace('_', ' ')):
66− results.append(first_char + '{engine}'.format(engine=engine.replace(' ', '_')))
67−
68− # check if query starts with engine shortcut
69− for engine_shortcut in engine_shortcuts:
70− if engine_shortcut.startswith(engine_query):
71− results.append(first_char + '{engine_shortcut}'.format(engine_shortcut=engine_shortcut))
72−
73− # check if current query stats with :bang
74− elif first_char == ':':
75− if len(full_query.getQuery()) == 1:
76− # show some example queries
77− results.append(":en")
78− results.append(":en_us")
79− results.append(":english")
80− results.append(":united_kingdom")
81− else:
82− engine_query = full_query.getQuery()[1:]
83−
84− for lc in language_codes:
85− lang_id, lang_name, country, english_name = map(str.lower, lc)
86−
87− # check if query starts with language-id
88− if lang_id.startswith(engine_query):
89− if len(engine_query) <= 2:
90− results.append(':{lang_id}'.format(lang_id=lang_id.split('-')[0]))
91− else:
92− results.append(':{lang_id}'.format(lang_id=lang_id))
93−
94− # check if query starts with language name
95− if lang_name.startswith(engine_query) or english_name.startswith(engine_query):
96− results.append(':{lang_name}'.format(lang_name=lang_name))
97−
98− # check if query starts with country
99− if country.startswith(engine_query.replace('_', ' ')):
100− results.append(':{country}'.format(country=country.replace(' ', '_')))
101−
102− # remove duplicates
103− result_set = set(results)
104−
105− # remove results which are already contained in the query
106− for query_part in full_query.query_parts:
107− if query_part in result_set:
108− result_set.remove(query_part)
109−
110− # convert result_set back to list
111− return list(result_set)
47+ if resp.ok:
48+ data = loads(resp.text)
49+ for suggestion in data[1]:
50+ results.append(suggestion)
51+
52+ return results
11253
11354
11455def dbpedia(query, lang):
11556 # dbpedia autocompleter, no HTTPS
116− autocomplete_url = 'http://lookup.dbpedia.org/api/search.asmx/KeywordSearch?'
57+ autocomplete_url = 'https://lookup.dbpedia.org/api/search.asmx/KeywordSearch?'
11758
11859 response = get(autocomplete_url + urlencode(dict(QueryString=query)))
11960
@@ -121,8 +62,7 @@ def dbpedia(query, lang):
12162
12263 if response.ok:
12364 dom = etree.fromstring(response.content)
124− results = dom.xpath('//a:Result/a:Label//text()',
125− namespaces={'a': 'http://lookup.dbpedia.org/'})
65+ results = dom.xpath('//Result/Label//text()')
12666
12767 return results
12868
@@ -154,12 +94,11 @@ def google(query, lang):
15494
15595def startpage(query, lang):
15696 # startpage autocompleter
157− url = 'https://startpage.com/do/suggest?{query}'
158−
159− resp = get(url.format(query=urlencode({'query': query}))).text.split('\n')
160− if len(resp) > 1:
161− return resp
162− return []
97+ lui = ENGINES_LANGUAGES['startpage'].get(lang, 'english')
98+ url = 'https://startpage.com/suggestions?{query}'
99+ resp = get(url.format(query=urlencode({'q': query, 'segment': 'startpage.udog', 'lui': lui})))
100+ data = resp.json()
101+ return [e['text'] for e in data.get('suggestions', []) if 'text' in e]
163102
164103
165104def swisscows(query, lang):
@@ -197,7 +136,8 @@ def wikipedia(query, lang):
197136 return []
198137
199138
200−backends = {'dbpedia': dbpedia,
139+backends = {'brave': brave,
140+ 'dbpedia': dbpedia,
201141 'duckduckgo': duckduckgo,
202142 'google': google,
203143 'startpage': startpage,
@@ -205,3 +145,14 @@ backends = {'dbpedia': dbpedia,
205145 'qwant': qwant,
206146 'wikipedia': wikipedia
207147 }
148+
149+
150+def search_autocomplete(backend_name, query, lang):
151+ backend = backends.get(backend_name)
152+ if backend is None:
153+ return []
154+
155+ try:
156+ return backend(query, lang)
157+ except (HTTPError, SearxEngineResponseException):
158+ return []
searx/brand.pydeleted−6
@@ -1,6 +0,0 @@
1−GIT_URL = 'https://github.com/searx/searx'
2−GIT_BRANCH = 'master'
3−ISSUE_URL = 'https://github.com/searx/searx/issues'
4−SEARX_URL = 'https://searx.me'
5−DOCS_URL = 'https://searx.github.io/searx'
6−PUBLIC_INSTANCES = 'https://searx.space'
searx/data/__init__.py+38−17
@@ -1,29 +1,50 @@
1+# SPDX-License-Identifier: AGPL-3.0-or-later
2+# lint: pylint
3+"""This module holds the *data* created by::
4+
5+ make data.all
6+
7+"""
8+
9+__all__ = [
10+ 'ENGINES_LANGUAGES',
11+ 'CURRENCIES',
12+ 'USER_AGENTS',
13+ 'EXTERNAL_URLS',
14+ 'WIKIDATA_UNITS',
15+ 'EXTERNAL_BANGS',
16+ 'OSM_KEYS_TAGS',
17+ 'ahmia_blacklist_loader',
18+]
19+
120import json
221from pathlib import Path
322
4−
5−__init__ = ['ENGINES_LANGUGAGES', 'CURRENCIES', 'USER_AGENTS', 'EXTERNAL_URLS', 'WIKIDATA_UNITS',
6− 'bangs_loader', 'ahmia_blacklist_loader']
723data_dir = Path(__file__).parent
824
925
10−def load(filename):
11− # add str(...) for Python 3.5
12− with open(str(data_dir / filename), encoding='utf-8') as fd:
13− return json.load(fd)
26+def _load(filename):
27+ with open(data_dir / filename, encoding='utf-8') as f:
28+ return json.load(f)
29+
1430
31+def ahmia_blacklist_loader():
32+ """Load data from `ahmia_blacklist.txt` and return a list of MD5 values of onion
33+ names. The MD5 values are fetched by::
1534
16−def bangs_loader():
17− return load('bangs.json')
35+ searx_extra/update/update_ahmia_blacklist.py
1836
37+ This function is used by :py:mod:`searx.plugins.ahmia_filter`.
1938
20−def ahmia_blacklist_loader():
21− with open(str(data_dir / 'ahmia_blacklist.txt'), encoding='utf-8') as fd:
22− return fd.read().split()
39+ """
40+ with open(str(data_dir / 'ahmia_blacklist.txt'), encoding='utf-8') as f:
41+ return f.read().split()
2342
2443
25−ENGINES_LANGUAGES = load('engines_languages.json')
26−CURRENCIES = load('currencies.json')
27−USER_AGENTS = load('useragents.json')
28−EXTERNAL_URLS = load('external_urls.json')
29−WIKIDATA_UNITS = load('wikidata_units.json')
44+ENGINES_LANGUAGES = _load('engines_languages.json')
45+CURRENCIES = _load('currencies.json')
46+USER_AGENTS = _load('useragents.json')
47+EXTERNAL_URLS = _load('external_urls.json')
48+WIKIDATA_UNITS = _load('wikidata_units.json')
49+EXTERNAL_BANGS = _load('external_bangs.json')
50+OSM_KEYS_TAGS = _load('osm_keys_tags.json')
searx/data/ahmia_blacklist.txt+19553
This diff is too big to show here.
searx/data/bangs.jsondeleted−67913
This diff is too big to show here.
searx/data/currencies.json+14643−7619
This diff is too big to show here.
searx/data/engines_languages.json+653−1483
This diff is too big to show here.
searx/data/external_bangs.jsonnew file+19068
This diff is too big to show here.
searx/data/osm_keys_tags.jsonnew file+15416
This diff is too big to show here.
searx/data/useragents.json+5−4
This diff is too big to show here.
searx/data/wikidata_units.json+275−58
This diff is too big to show here.
searx/engines/1337x.py+22−8
This diff is too big to show here.
searx/engines/__init__.py+42−22
This diff is too big to show here.
searx/engines/acgsou.pydeleted−74
This diff is too big to show here.
searx/engines/ahmia.py+19−16
This diff is too big to show here.
searx/engines/apkmirror.py+31−25
This diff is too big to show here.
searx/engines/archlinux.py+15−12
This diff is too big to show here.
searx/engines/arxiv.py+22−24
This diff is too big to show here.
searx/engines/bandcamp.pynew file+73
This diff is too big to show here.
searx/engines/base.py+11−15
This diff is too big to show here.
searx/engines/bing.py+13−15
This diff is too big to show here.
searx/engines/bing_images.py+14−14
This diff is too big to show here.
searx/engines/bing_news.py+20−17
This diff is too big to show here.
searx/engines/bing_videos.py+13−10
This diff is too big to show here.
searx/engines/btdigg.py+14−9
This diff is too big to show here.
searx/engines/ccengine.pynew file+53
This diff is too big to show here.
searx/engines/command.py+9−21
This diff is too big to show here.
searx/engines/core.pynew file+82
This diff is too big to show here.
searx/engines/currency_convert.py+18−46
This diff is too big to show here.
searx/engines/dailymotion.py+11−11
This diff is too big to show here.
searx/engines/deezer.py+11−8
This diff is too big to show here.
searx/engines/deviantart.py+49−54
This diff is too big to show here.
searx/engines/dictzone.py+19−27
This diff is too big to show here.
searx/engines/digbt.py+10−8
This diff is too big to show here.
searx/engines/digg.py+44−44
This diff is too big to show here.
searx/engines/doku.py+14−11
This diff is too big to show here.
searx/engines/duckduckgo.py+112−57
This diff is too big to show here.
searx/engines/duckduckgo_definitions.py+17−11
This diff is too big to show here.
searx/engines/duckduckgo_images.py+20−23
This diff is too big to show here.
searx/engines/duden.py+30−32
This diff is too big to show here.
searx/engines/dummy-offline.py+11−3
This diff is too big to show here.
searx/engines/dummy.py+11−3
This diff is too big to show here.
searx/engines/ebay.py+14−9
This diff is too big to show here.
searx/engines/elasticsearch.pynew file+144
This diff is too big to show here.
searx/engines/etools.py+11−8
This diff is too big to show here.
searx/engines/fdroid.py+11−7
This diff is too big to show here.
searx/engines/filecrop.pydeleted−85
This diff is too big to show here.
searx/engines/flickr.py+11−9
This diff is too big to show here.
searx/engines/flickr_noapi.py+12−11
This diff is too big to show here.
searx/engines/framalibre.py+11−8
This diff is too big to show here.
searx/engines/freesound.pynew file+69
This diff is too big to show here.
searx/engines/frinkiac.py+15−8
This diff is too big to show here.
searx/engines/genius.py+18−17
This diff is too big to show here.
searx/engines/gentoo.py+11−10
This diff is too big to show here.
searx/engines/gigablast.py+35−23
This diff is too big to show here.
searx/engines/github.py+12−9
This diff is too big to show here.
searx/engines/google.py+139−94
This diff is too big to show here.
searx/engines/google_images.py+80−95
This diff is too big to show here.
searx/engines/google_news.py+167−57
This diff is too big to show here.
searx/engines/google_scholar.pynew file+156
This diff is too big to show here.
searx/engines/google_videos.py+173−69
This diff is too big to show here.
searx/engines/imdb.pynew file+55
This diff is too big to show here.
searx/engines/ina.py+16−18
This diff is too big to show here.
searx/engines/invidious.py+29−14
This diff is too big to show here.
searx/engines/json_engine.py+17−5
This diff is too big to show here.
searx/engines/kickass.py+11−8
This diff is too big to show here.
searx/engines/loc.pynew file+68
This diff is too big to show here.
searx/engines/mediathekviewweb.pynew file+68
This diff is too big to show here.
searx/engines/mediawiki.py+12−12
This diff is too big to show here.
searx/engines/meilisearch.pynew file+59
This diff is too big to show here.
searx/engines/microsoft_academic.py+37−40
This diff is too big to show here.
searx/engines/mixcloud.py+11−8
This diff is too big to show here.
searx/engines/mongodb.pynew file+61
This diff is too big to show here.
searx/engines/mysql_server.pynew file+62
This diff is too big to show here.
searx/engines/not_evil.pydeleted−64
This diff is too big to show here.
searx/engines/nyaa.py+11−7
This diff is too big to show here.
searx/engines/onesearch.pynew file+57
This diff is too big to show here.
searx/engines/opensemantic.py+13−9
This diff is too big to show here.
searx/engines/openstreetmap.py+407−74
This diff is too big to show here.
searx/engines/pdbe.py+11−8
This diff is too big to show here.
searx/engines/peertube.py+21−20
This diff is too big to show here.
searx/engines/photon.py+12−10
This diff is too big to show here.
searx/engines/piratebay.py+16−11
This diff is too big to show here.
searx/engines/postgresql.pynew file+69
This diff is too big to show here.
searx/engines/prowlarr.pynew file+80
This diff is too big to show here.
searx/engines/pubmed.py+16−12
This diff is too big to show here.
searx/engines/qwant.py+199−87
This diff is too big to show here.
searx/engines/recoll.pynew file+111
This diff is too big to show here.
searx/engines/reddit.py+11−8
This diff is too big to show here.
searx/engines/redis_server.pynew file+69
This diff is too big to show here.
searx/engines/rumble.pynew file+81
This diff is too big to show here.
searx/engines/scanr_structures.py+12−9
This diff is too big to show here.
searx/engines/searchcode_code.py+11−9
This diff is too big to show here.
searx/engines/searchcode_doc.pydeleted−49
This diff is too big to show here.
searx/engines/searx_engine.py+10−8
This diff is too big to show here.
searx/engines/seedpeer.pydeleted−78
This diff is too big to show here.
searx/engines/semantic_scholar.pynew file+42
This diff is too big to show here.
searx/engines/sepiasearch.py+14−9
This diff is too big to show here.
searx/engines/seznam.pynew file+66
This diff is too big to show here.
searx/engines/sjp.pynew file+92
This diff is too big to show here.
searx/engines/solidtorrents.pynew file+53
This diff is too big to show here.
searx/engines/solr.pynew file+74
This diff is too big to show here.
searx/engines/soundcloud.py+14−13
This diff is too big to show here.
searx/engines/spotify.py+14−10
This diff is too big to show here.
searx/engines/springer.pynew file+74
This diff is too big to show here.
searx/engines/sqlite.pynew file+51
This diff is too big to show here.
searx/engines/stackexchange.pynew file+67
This diff is too big to show here.
searx/engines/stackoverflow.pydeleted−57
This diff is too big to show here.
searx/engines/startpage.py+114−33
This diff is too big to show here.
searx/engines/tineye.pynew file+86
This diff is too big to show here.
searx/engines/tokyotoshokan.py+11−8
This diff is too big to show here.
searx/engines/torrentz.py+11−9
This diff is too big to show here.
searx/engines/translated.py+16−29
This diff is too big to show here.
searx/engines/twitter.pydeleted−87
This diff is too big to show here.
searx/engines/unsplash.py+11−8
This diff is too big to show here.
searx/engines/vimeo.py+14−13
This diff is too big to show here.
searx/engines/wikidata.py+23−17
This diff is too big to show here.
searx/engines/wikipedia.py+35−12
This diff is too big to show here.
searx/engines/wolframalpha_api.py+14−9
This diff is too big to show here.
searx/engines/wolframalpha_noapi.py+15−10
This diff is too big to show here.
searx/engines/wordnik.pynew file+77
This diff is too big to show here.
searx/engines/www1x.py+20−21
This diff is too big to show here.
searx/engines/xpath.py+20−15
This diff is too big to show here.
searx/engines/xpath_flex.pynew file+121
This diff is too big to show here.
searx/engines/yacy.py+16−16
This diff is too big to show here.
searx/engines/yahoo.py+104−101
This diff is too big to show here.
searx/engines/yahoo_news.py+84−82
This diff is too big to show here.
searx/engines/yandex.pydeleted−64
This diff is too big to show here.
searx/engines/yggtorrent.py+18−13
This diff is too big to show here.
searx/engines/youtube_api.py+16−11
This diff is too big to show here.
searx/engines/youtube_noapi.py+85−18
This diff is too big to show here.
searx/exceptions.py+70
This diff is too big to show here.
searx/external_bang.py+76−26
This diff is too big to show here.
searx/external_urls.py+14−2
This diff is too big to show here.
searx/languages.py+1−3
This diff is too big to show here.
searx/metrology/__init__.pynew file
No changes to the content.
searx/metrology/error_recorder.pynew file+144
This diff is too big to show here.
searx/network/__init__.pynew file+188
This diff is too big to show here.
searx/network/client.pynew file+219
This diff is too big to show here.
searx/network/network.pynew file+358
This diff is too big to show here.
searx/network/raise_for_httperror.pynew file+66
This diff is too big to show here.
searx/plugins/__init__.py+4−3
This diff is too big to show here.
searx/plugins/hostname_replace.pynew file+28
This diff is too big to show here.
searx/plugins/oa_doi_rewrite.py+2−2
This diff is too big to show here.
searx/plugins/self_info.py+1−1
This diff is too big to show here.
searx/plugins/tracker_url_remover.py+1
This diff is too big to show here.
searx/poolrequests.pydeleted−184
This diff is too big to show here.
searx/preferences.py+19−8
This diff is too big to show here.
searx/query.py+286−118
This diff is too big to show here.
searx/results.py+36−9
This diff is too big to show here.
searx/search.pydeleted−467
This diff is too big to show here.
searx/search/__init__.pynew file+216
This diff is too big to show here.
searx/search/checker/__init__.pynew file+4
This diff is too big to show here.
searx/search/checker/__main__.pynew file+95
This diff is too big to show here.
searx/search/checker/background.pynew file+125
This diff is too big to show here.
searx/search/checker/impl.pynew file+416
This diff is too big to show here.
searx/search/models.pynew file+71
This diff is too big to show here.
searx/search/processors/__init__.pynew file+41
This diff is too big to show here.
searx/search/processors/abstract.pynew file+52
This diff is too big to show here.
searx/search/processors/offline.pynew file+51
This diff is too big to show here.
searx/search/processors/online.pynew file+262
This diff is too big to show here.
searx/search/processors/online_currency.pynew file+69
This diff is too big to show here.
searx/search/processors/online_dictionary.pynew file+55
This diff is too big to show here.
searx/settings.yml+858−159
This diff is too big to show here.
searx/settings_loader.pynew file+132
This diff is too big to show here.
searx/settings_robot.yml+8−2
This diff is too big to show here.
searx/shared/__init__.pynew file+31
This diff is too big to show here.
searx/shared/shared_abstract.pynew file+21
This diff is too big to show here.
searx/shared/shared_simple.pynew file+39
This diff is too big to show here.
searx/shared/shared_uwsgi.pynew file+64
This diff is too big to show here.
searx/static/css/bootstrap.min.cssdeleted−1
This diff is too big to show here.
searx/static/fonts/glyphicons-halflings-regular.eotdeleted
Binary file not shown.
searx/static/fonts/glyphicons-halflings-regular.svgdeleted−229
This diff is too big to show here.
searx/static/fonts/glyphicons-halflings-regular.ttfdeleted
Binary file not shown.
searx/static/fonts/glyphicons-halflings-regular.woffdeleted
Binary file not shown.
searx/static/js/bootstrap.min.jsdeleted−6
This diff is too big to show here.
searx/static/js/html5shiv.min.jsdeleted−4
This diff is too big to show here.
searx/static/js/jquery-1.11.1.min.jsdeleted−4
This diff is too big to show here.
searx/static/js/mootools-autocompleter-1.1.2-min.jsdeleted−2
This diff is too big to show here.
searx/static/js/mootools-core-1.4.5-min.jsdeleted−491
This diff is too big to show here.
searx/static/js/require-2.1.15.min.jsdeleted−36
This diff is too big to show here.
searx/static/js/respond.min.jsdeleted−5
This diff is too big to show here.
searx/static/js/typeahead.bundle.min.jsdeleted−7
This diff is too big to show here.
searx/static/less/bootstrap/.csscomb.jsondeleted−297
This diff is too big to show here.
searx/static/less/bootstrap/.csslintrcdeleted−19
This diff is too big to show here.
searx/static/less/bootstrap/alerts.lessdeleted−68
This diff is too big to show here.
searx/static/less/bootstrap/badges.lessdeleted−55
This diff is too big to show here.
searx/static/less/bootstrap/bootstrap.lessdeleted−53
This diff is too big to show here.
searx/static/less/bootstrap/breadcrumbs.lessdeleted−26
This diff is too big to show here.
searx/static/less/bootstrap/button-groups.lessdeleted−240
This diff is too big to show here.
searx/static/less/bootstrap/buttons.lessdeleted−157
This diff is too big to show here.
searx/static/less/bootstrap/carousel.lessdeleted−243
This diff is too big to show here.
searx/static/less/bootstrap/close.lessdeleted−33
This diff is too big to show here.
searx/static/less/bootstrap/code.lessdeleted−68
This diff is too big to show here.
searx/static/less/bootstrap/component-animations.lessdeleted−31
This diff is too big to show here.
searx/static/less/bootstrap/dropdowns.lessdeleted−215
This diff is too big to show here.
searx/static/less/bootstrap/forms.lessdeleted−540
This diff is too big to show here.
searx/static/less/bootstrap/glyphicons.lessdeleted−233
This diff is too big to show here.
searx/static/less/bootstrap/grid.lessdeleted−84
This diff is too big to show here.
searx/static/less/bootstrap/input-groups.lessdeleted−166
This diff is too big to show here.
searx/static/less/bootstrap/jumbotron.lessdeleted−48
This diff is too big to show here.
searx/static/less/bootstrap/labels.lessdeleted−64
This diff is too big to show here.
searx/static/less/bootstrap/list-group.lessdeleted−131
This diff is too big to show here.
searx/static/less/bootstrap/media.lessdeleted−56
This diff is too big to show here.
searx/static/less/bootstrap/mixins.lessdeleted−39
This diff is too big to show here.
searx/static/less/bootstrap/mixins/alerts.lessdeleted−14
This diff is too big to show here.
searx/static/less/bootstrap/mixins/background-variant.lessdeleted−8
This diff is too big to show here.
searx/static/less/bootstrap/mixins/border-radius.lessdeleted−18
This diff is too big to show here.
searx/static/less/bootstrap/mixins/buttons.lessdeleted−50
This diff is too big to show here.
searx/static/less/bootstrap/mixins/center-block.lessdeleted−7
This diff is too big to show here.
searx/static/less/bootstrap/mixins/clearfix.lessdeleted−22
This diff is too big to show here.
searx/static/less/bootstrap/mixins/forms.lessdeleted−81
This diff is too big to show here.
searx/static/less/bootstrap/mixins/gradients.lessdeleted−59
This diff is too big to show here.
searx/static/less/bootstrap/mixins/grid-framework.lessdeleted−91
This diff is too big to show here.
searx/static/less/bootstrap/mixins/grid.lessdeleted−122
This diff is too big to show here.
searx/static/less/bootstrap/mixins/hide-text.lessdeleted−21
This diff is too big to show here.
searx/static/less/bootstrap/mixins/image.lessdeleted−34
This diff is too big to show here.
searx/static/less/bootstrap/mixins/labels.lessdeleted−12
This diff is too big to show here.
searx/static/less/bootstrap/mixins/list-group.lessdeleted−29
This diff is too big to show here.
searx/static/less/bootstrap/mixins/nav-divider.lessdeleted−10
This diff is too big to show here.
searx/static/less/bootstrap/mixins/nav-vertical-align.lessdeleted−9
This diff is too big to show here.
searx/static/less/bootstrap/mixins/opacity.lessdeleted−8
This diff is too big to show here.
searx/static/less/bootstrap/mixins/pagination.lessdeleted−23
This diff is too big to show here.
searx/static/less/bootstrap/mixins/panels.lessdeleted−24
This diff is too big to show here.
searx/static/less/bootstrap/mixins/progress-bar.lessdeleted−10
This diff is too big to show here.
searx/static/less/bootstrap/mixins/reset-filter.lessdeleted−8
This diff is too big to show here.
searx/static/less/bootstrap/mixins/resize.lessdeleted−6
This diff is too big to show here.
searx/static/less/bootstrap/mixins/responsive-visibility.lessdeleted−15
This diff is too big to show here.
searx/static/less/bootstrap/mixins/size.lessdeleted−10
This diff is too big to show here.
searx/static/less/bootstrap/mixins/tab-focus.lessdeleted−9
This diff is too big to show here.
searx/static/less/bootstrap/mixins/table-row.lessdeleted−28
This diff is too big to show here.
searx/static/less/bootstrap/mixins/text-emphasis.lessdeleted−8
This diff is too big to show here.
searx/static/less/bootstrap/mixins/text-overflow.lessdeleted−8
This diff is too big to show here.
searx/static/less/bootstrap/mixins/vendor-prefixes.lessdeleted−224
This diff is too big to show here.
searx/static/less/bootstrap/modals.lessdeleted−150
This diff is too big to show here.
searx/static/less/bootstrap/navbar.lessdeleted−655
This diff is too big to show here.
searx/static/less/bootstrap/navs.lessdeleted−242
This diff is too big to show here.
searx/static/less/bootstrap/normalize.lessdeleted−425
This diff is too big to show here.
searx/static/less/bootstrap/pager.lessdeleted−55
This diff is too big to show here.
searx/static/less/bootstrap/pagination.lessdeleted−88
This diff is too big to show here.
searx/static/less/bootstrap/panels.lessdeleted−243
This diff is too big to show here.
searx/static/less/bootstrap/popovers.lessdeleted−133
This diff is too big to show here.
searx/static/less/bootstrap/print.lessdeleted−101
This diff is too big to show here.
searx/static/less/bootstrap/progress-bars.lessdeleted−105
This diff is too big to show here.
searx/static/less/bootstrap/responsive-embed.lessdeleted−34
This diff is too big to show here.
searx/static/less/bootstrap/responsive-utilities.lessdeleted−194
This diff is too big to show here.
searx/static/less/bootstrap/scaffolding.lessdeleted−150
This diff is too big to show here.
searx/static/less/bootstrap/tables.lessdeleted−233
This diff is too big to show here.
searx/static/less/bootstrap/theme.lessdeleted−258
This diff is too big to show here.
searx/static/less/bootstrap/thumbnails.lessdeleted−36
This diff is too big to show here.
searx/static/less/bootstrap/tooltip.lessdeleted−95
This diff is too big to show here.
searx/static/less/bootstrap/type.lessdeleted−313
This diff is too big to show here.
searx/static/less/bootstrap/typeahead.lessdeleted−153
This diff is too big to show here.
searx/static/less/bootstrap/utilities.lessdeleted−57
This diff is too big to show here.
searx/static/less/bootstrap/variables.lessdeleted−846
This diff is too big to show here.
searx/static/less/bootstrap/wells.lessdeleted−29
This diff is too big to show here.
searx/static/plugins/js/infinite_scroll.js+1−1
This diff is too big to show here.
searx/static/themes/simple/js/searx_src/searx_imageresult.js → searx/static/themes/__common__/js/image_layout.jsrenamed+61−49
This diff is too big to show here.
searx/static/themes/courgette/css/style-rtl.cssdeleted−1
This diff is too big to show here.
searx/static/themes/courgette/css/style.cssdeleted−1
This diff is too big to show here.
searx/static/themes/courgette/img/bg-body-index.jpgdeleted
Binary file not shown.
searx/static/themes/courgette/img/favicon.pngdeleted
Binary file not shown.
searx/static/themes/courgette/img/github_ribbon.pngdeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_500px.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_bing.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_dailymotion.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_deezer.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_deviantart.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_digg.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_duckduckgo.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_flickr.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_github.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_google play apps.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_google play movies.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_google play music.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_google.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_kickass.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_openstreetmap.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_searchcode code.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_searchcode doc.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_searchcode.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_soundcloud.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_stackoverflow.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_startpage.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_subtitleseeker.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_twitter.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_vimeo.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_wikipedia.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_yahoo.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_youtube.icodeleted
Binary file not shown.
searx/static/themes/courgette/img/preference-icon.pngdeleted
Binary file not shown.
searx/static/themes/courgette/img/search-icon.pngdeleted
Binary file not shown.
searx/static/themes/courgette/img/searx-mobile.pngdeleted
Binary file not shown.
searx/static/themes/courgette/img/searx.pngdeleted
Binary file not shown.
searx/static/themes/courgette/img/searx_logo.svgdeleted−203
This diff is too big to show here.
searx/static/themes/courgette/js/searx.jsdeleted−45
This diff is too big to show here.
searx/static/themes/courgette/less/style-rtl.lessdeleted−42
This diff is too big to show here.
searx/static/themes/courgette/less/style.lessdeleted−695
This diff is too big to show here.
searx/static/themes/legacy/css/style-rtl.cssdeleted−1
This diff is too big to show here.
searx/static/themes/legacy/css/style.cssdeleted−1
This diff is too big to show here.
searx/static/themes/legacy/img/favicon.pngdeleted
Binary file not shown.
searx/static/themes/legacy/img/github_ribbon.pngdeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_500px.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_bing.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_dailymotion.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_deezer.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_deviantart.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_digg.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_duckduckgo.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_flickr.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_github.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_google play apps.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_google play movies.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_google play music.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_google.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_kickass.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_openstreetmap.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_searchcode code.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_searchcode doc.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_searchcode.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_soundcloud.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_stackoverflow.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_startpage.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_subtitleseeker.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_twitter.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_vimeo.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_wikipedia.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_yahoo.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_youtube.icodeleted
Binary file not shown.
searx/static/themes/legacy/img/preference-icon.pngdeleted
Binary file not shown.
searx/static/themes/legacy/img/search-icon.pngdeleted
Binary file not shown.
searx/static/themes/legacy/img/searx.pngdeleted
Binary file not shown.
searx/static/themes/legacy/img/searx_logo.svgdeleted−203
This diff is too big to show here.
searx/static/themes/legacy/js/searx.jsdeleted−49
This diff is too big to show here.
searx/static/themes/legacy/less/autocompleter.lessdeleted−61
This diff is too big to show here.
searx/static/themes/legacy/less/code.lessdeleted−83
This diff is too big to show here.
searx/static/themes/legacy/less/definitions.lessdeleted−119
This diff is too big to show here.
searx/static/themes/legacy/less/mixins.lessdeleted−27
This diff is too big to show here.
searx/static/themes/legacy/less/search.lessdeleted−68
This diff is too big to show here.
searx/static/themes/legacy/less/style-rtl.lessdeleted−11
This diff is too big to show here.
searx/static/themes/legacy/less/style.lessdeleted−747
This diff is too big to show here.
searx/static/themes/oscar/README.rst+39
This diff is too big to show here.
searx/static/themes/oscar/css/bootstrap-theme.cssnew file+587
This diff is too big to show here.
searx/static/themes/oscar/css/bootstrap-theme.min.cssnew file+6
This diff is too big to show here.
searx/static/themes/oscar/css/bootstrap-theme.min.css.mapnew file+1
This diff is too big to show here.
searx/static/themes/oscar/css/bootstrap.cssnew file+6990
This diff is too big to show here.
searx/static/themes/oscar/css/bootstrap.min.cssnew file+5
This diff is too big to show here.
searx/static/themes/oscar/css/bootstrap.min.css.mapnew file+1
This diff is too big to show here.
searx/static/themes/oscar/css/images/layers-2x.pngnew file
Binary file not shown.
searx/static/themes/oscar/css/images/layers.pngnew file
Binary file not shown.
searx/static/themes/oscar/css/images/marker-icon-2x.pngnew file
Binary file not shown.
searx/static/themes/oscar/css/images/marker-icon.pngnew file
Binary file not shown.
searx/static/themes/oscar/css/images/marker-shadow.pngnew file
Binary file not shown.
searx/static/themes/oscar/css/leaflet.cssnew file+640
This diff is too big to show here.
searx/static/themes/oscar/css/leaflet.min.css+1−93
This diff is too big to show here.
searx/static/themes/oscar/css/leaflet.min.css.mapnew file+1
This diff is too big to show here.
searx/static/themes/oscar/css/logicodev-dark.css+283−175
This diff is too big to show here.
searx/static/themes/oscar/css/logicodev-dark.min.css+1−1
This diff is too big to show here.
searx/static/themes/oscar/css/logicodev-dark.min.css.mapnew file+1
This diff is too big to show here.
searx/static/themes/oscar/css/logicodev.css+296−178
This diff is too big to show here.
searx/static/themes/oscar/css/logicodev.min.css+1−1
This diff is too big to show here.
searx/static/themes/oscar/css/logicodev.min.css.mapnew file+1
This diff is too big to show here.
searx/static/themes/oscar/css/pointhi.css+185−104
This diff is too big to show here.
searx/static/themes/oscar/css/pointhi.min.css+1−1
This diff is too big to show here.
searx/static/themes/oscar/css/pointhi.min.css.mapnew file+1
This diff is too big to show here.
searx/static/themes/oscar/fonts/glyphicons-halflings-regular.eotnew file
Binary file not shown.
searx/static/themes/oscar/fonts/glyphicons-halflings-regular.svgnew file+288
This diff is too big to show here.
searx/static/themes/oscar/fonts/glyphicons-halflings-regular.ttfnew file
Binary file not shown.
searx/static/themes/oscar/fonts/glyphicons-halflings-regular.woffnew file
Binary file not shown.
searx/static/themes/oscar/fonts/glyphicons-halflings-regular.woff2new file
Binary file not shown.
searx/static/themes/oscar/gruntfile.js+87−29
This diff is too big to show here.
searx/static/themes/oscar/img/icons/bandcamp.pngnew file
Binary file not shown.
searx/static/themes/oscar/img/map/layers-2x.pngdeleted
Binary file not shown.
searx/static/themes/oscar/img/map/layers.pngdeleted
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon-2x-green.pngdeleted
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon-2x-orange.pngdeleted
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon-2x-red.pngdeleted
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon-2x.pngdeleted
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon-green.pngdeleted
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon-orange.pngdeleted
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon-red.pngdeleted
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon.pngdeleted
Binary file not shown.
searx/static/themes/oscar/img/map/marker-shadow.pngdeleted
Binary file not shown.
searx/static/themes/oscar/js/bootstrap.bundle.min.jsnew file+7
This diff is too big to show here.
searx/static/themes/oscar/js/bootstrap.min.jsnew file+6
This diff is too big to show here.
searx/static/themes/oscar/js/jquery.min.jsnew file+2
This diff is too big to show here.
searx/static/themes/oscar/js/leaflet-0.7.3.min.jsdeleted−9
This diff is too big to show here.
searx/static/themes/oscar/js/leaflet.jsnew file+6
This diff is too big to show here.
searx/static/themes/oscar/js/searx.js+238−158
This diff is too big to show here.
searx/static/themes/oscar/js/searx.min.js+4−2
This diff is too big to show here.
searx/static/themes/oscar/js/searx.min.js.mapnew file+1
This diff is too big to show here.
searx/static/themes/oscar/js/searx_src/00_requirejs_config.jsdeleted−23
This diff is too big to show here.
searx/static/themes/oscar/js/searx_src/leaflet_map.jsdeleted−167
This diff is too big to show here.
searx/static/themes/oscar/js/typeahead.bundle.min.js+9
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev/code.less−101
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev/preferences.less−15
This diff is too big to show here.
searx/static/themes/oscar/less/pointhi/code.less−79
This diff is too big to show here.
searx/static/themes/oscar/less/pointhi/preferences.less−3
This diff is too big to show here.
searx/static/themes/oscar/package.json+12−7
This diff is too big to show here.
searx/static/themes/oscar/src/js/01_init.js+5−1
This diff is too big to show here.
searx/static/themes/oscar/src/js/autocompleter.js+24−13
This diff is too big to show here.
searx/static/themes/oscar/src/js/element_modifiers.js+13
This diff is too big to show here.
searx/static/themes/oscar/src/js/infobox.js
This diff is too big to show here.
searx/static/themes/oscar/src/js/leaflet_map.js+79
This diff is too big to show here.
searx/static/themes/oscar/src/js/toggleall.js
This diff is too big to show here.
searx/static/themes/oscar/src/less/bootstrap/bootstrap.less+3
This diff is too big to show here.
searx/static/themes/oscar/src/less/bootstrap/typeahead.less+154
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev-dark/oscar.less+16−4
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/advanced.less
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/checkbox.less
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/code.less+14
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/cursor.less
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/footer.less
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/infobox.less+5
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/navbar.less
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/onoff.less
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/oscar.less
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/preferences.less+31
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/pygments.less+99
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/results.less+81−7
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/search.less+5−10
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/variables.less
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/advanced.less
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/checkbox.less
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/code.less+2
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/cursor.less
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/footer.less
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/infobox.less
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/navbar.less
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/onoff.less
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/oscar.less+2
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/preferences.less+19
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/pygments.less+99
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/results.less+4
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/search.less
This diff is too big to show here.
searx/static/themes/pix-art/css/style.css−1
This diff is too big to show here.
searx/static/themes/pix-art/img/favicon.png
This diff is too big to show here.
searx/static/themes/pix-art/img/preference-icon-pixel.png
This diff is too big to show here.
searx/static/themes/pix-art/img/search-icon-pixel.png
This diff is too big to show here.
searx/static/themes/pix-art/img/searx-pixel-small.png
This diff is too big to show here.
searx/static/themes/pix-art/img/searx-pixel.png
This diff is too big to show here.
searx/static/themes/pix-art/js/searx.js−141
This diff is too big to show here.
searx/static/themes/pix-art/less/definitions.less−119
This diff is too big to show here.
searx/static/themes/pix-art/less/mixins.less−27
This diff is too big to show here.
searx/static/themes/pix-art/less/search.less−57
This diff is too big to show here.
searx/static/themes/pix-art/less/style.less−451
This diff is too big to show here.
searx/static/themes/simple/css/searx-rtl.css+164−83
This diff is too big to show here.
searx/static/themes/simple/css/searx-rtl.min.css+1−1
This diff is too big to show here.
searx/static/themes/simple/css/searx.css+164−83
This diff is too big to show here.
searx/static/themes/simple/css/searx.min.css+1−1
This diff is too big to show here.
searx/static/themes/simple/gruntfile.js+2−2
This diff is too big to show here.
searx/static/themes/simple/js/searx.head.js+1−1
This diff is too big to show here.
searx/static/themes/simple/js/searx.head.min.js+2−2
This diff is too big to show here.
searx/static/themes/simple/js/searx.head.min.js.map+1−1
This diff is too big to show here.
searx/static/themes/simple/js/searx.js+165−240
This diff is too big to show here.
searx/static/themes/simple/js/searx.min.js+5−4
This diff is too big to show here.
searx/static/themes/simple/js/searx.min.js.map+1−1
This diff is too big to show here.
searx/static/themes/simple/js/searx_head/00_init.js+1−1
This diff is too big to show here.
searx/static/themes/simple/js/searx_src/searx_mapresult.js−87
This diff is too big to show here.
searx/static/themes/simple/js/searx_src/searx_results.js+1−1
This diff is too big to show here.
searx/static/themes/simple/js/searx_src/searx_search.js+1−1
This diff is too big to show here.
searx/static/themes/simple/less/code.less+5−80
This diff is too big to show here.
searx/static/themes/simple/less/preferences.less+17
This diff is too big to show here.
searx/static/themes/simple/less/pygments.less+99
This diff is too big to show here.
searx/static/themes/simple/less/style.less+32
This diff is too big to show here.
searx/templates/__common__/opensearch.xml+4−4
This diff is too big to show here.
searx/templates/__common__/opensearch_response_rss.xml+1−1
This diff is too big to show here.
searx/templates/__common__/translations.js.tpl−1
This diff is too big to show here.
searx/templates/courgette/404.html−9
This diff is too big to show here.
searx/templates/courgette/about.html−5
This diff is too big to show here.
searx/templates/courgette/base.html−43
This diff is too big to show here.
searx/templates/courgette/categories.html−9
This diff is too big to show here.
searx/templates/courgette/color.css−34
This diff is too big to show here.
searx/templates/courgette/github_ribbon.html−3
This diff is too big to show here.
searx/templates/courgette/index.html−17
This diff is too big to show here.
searx/templates/courgette/preferences.html−150
This diff is too big to show here.
searx/templates/courgette/result_templates/code.html−11
This diff is too big to show here.
searx/templates/courgette/result_templates/default.html−13
This diff is too big to show here.
searx/templates/courgette/result_templates/images.html−6
This diff is too big to show here.
searx/templates/courgette/result_templates/key-value.html−13
This diff is too big to show here.
searx/templates/courgette/result_templates/map.html−13
This diff is too big to show here.
searx/templates/courgette/result_templates/torrent.html−13
This diff is too big to show here.
searx/templates/courgette/result_templates/videos.html−10
This diff is too big to show here.
searx/templates/courgette/results.html−87
This diff is too big to show here.
searx/templates/courgette/search.html−7
This diff is too big to show here.
searx/templates/courgette/stats.html−22
This diff is too big to show here.
searx/templates/legacy/404.html−9
This diff is too big to show here.
searx/templates/legacy/about.html−5
This diff is too big to show here.
searx/templates/legacy/base.html−38
This diff is too big to show here.
searx/templates/legacy/categories.html−10
This diff is too big to show here.
searx/templates/legacy/github_ribbon.html−3
This diff is too big to show here.
searx/templates/legacy/index.html−18
This diff is too big to show here.
searx/templates/legacy/infobox.html−51
This diff is too big to show here.
searx/templates/legacy/preferences.html−147
This diff is too big to show here.
searx/templates/legacy/result_templates/code.html−11
This diff is too big to show here.
searx/templates/legacy/result_templates/default.html−11
This diff is too big to show here.
searx/templates/legacy/result_templates/images.html−6
This diff is too big to show here.
searx/templates/legacy/result_templates/key-value.html−13
This diff is too big to show here.
searx/templates/legacy/result_templates/map.html−13
This diff is too big to show here.
searx/templates/legacy/result_templates/torrent.html−13
This diff is too big to show here.
searx/templates/legacy/result_templates/videos.html−6
This diff is too big to show here.
searx/templates/legacy/results.html−104
This diff is too big to show here.
searx/templates/legacy/search.html−8
This diff is too big to show here.
searx/templates/legacy/stats.html−22
This diff is too big to show here.
searx/templates/oscar/base.html+14−15
This diff is too big to show here.
searx/templates/oscar/index.html+7−8
This diff is too big to show here.
searx/templates/oscar/macros.html+70−44
This diff is too big to show here.
searx/templates/oscar/messages/no_results.html+1−1
This diff is too big to show here.
searx/templates/oscar/preferences.html+33−10
This diff is too big to show here.
searx/templates/oscar/result_templates/default.html+7−7
This diff is too big to show here.
searx/templates/oscar/result_templates/files.html+55
This diff is too big to show here.
searx/templates/oscar/result_templates/key-value.html+2−2
This diff is too big to show here.
searx/templates/oscar/result_templates/map.html+60−56
This diff is too big to show here.
searx/templates/oscar/result_templates/products.html+2−1
This diff is too big to show here.
searx/templates/oscar/result_templates/torrent.html+15−15
This diff is too big to show here.
searx/templates/oscar/result_templates/videos.html+16−16
This diff is too big to show here.
searx/templates/oscar/results.html+20−9
This diff is too big to show here.
searx/templates/oscar/search.html+2−2
This diff is too big to show here.
searx/templates/oscar/search_full.html+1−1
This diff is too big to show here.
searx/templates/pix-art/404.html−9
This diff is too big to show here.
searx/templates/pix-art/about.html−4
This diff is too big to show here.
searx/templates/pix-art/base.html−35
This diff is too big to show here.
searx/templates/pix-art/index.html−12
This diff is too big to show here.
searx/templates/pix-art/preferences.html−90
This diff is too big to show here.
searx/templates/pix-art/result_templates/default.html−7
This diff is too big to show here.
searx/templates/pix-art/result_templates/images.html−6
This diff is too big to show here.
searx/templates/pix-art/results.html−32
This diff is too big to show here.
searx/templates/pix-art/search.html−9
This diff is too big to show here.
searx/templates/pix-art/stats.html−22
This diff is too big to show here.
searx/templates/simple/base.html+3−3
This diff is too big to show here.
searx/templates/simple/infobox.html+1−1
This diff is too big to show here.
searx/templates/simple/macros.html+2−2
This diff is too big to show here.
searx/templates/simple/messages/no_results.html+1−1
This diff is too big to show here.
searx/templates/simple/preferences.html+14−3
This diff is too big to show here.
searx/templates/simple/result_templates/map.html+35−52
This diff is too big to show here.
searx/templates/simple/results.html+16−7
This diff is too big to show here.
searx/templates/simple/search.html+1−1
This diff is too big to show here.
searx/testing.py+3−3
This diff is too big to show here.
searx/translations/ar/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/bg/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/bo/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/ca/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/cs/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/cs/LC_MESSAGES/messages.po+93−92
This diff is too big to show here.
searx/translations/cy/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/da/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/de/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/de/LC_MESSAGES/messages.po+21−19
This diff is too big to show here.
searx/translations/el_GR/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/en/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/eo/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/es/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/es/LC_MESSAGES/messages.po+20−19
This diff is too big to show here.
searx/translations/et/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/et/LC_MESSAGES/messages.po+18−18
This diff is too big to show here.
searx/translations/eu/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/eu/LC_MESSAGES/messages.po+22−21
This diff is too big to show here.
searx/translations/fa_IR/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/fi/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/fil/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/fr/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/fr/LC_MESSAGES/messages.po+21−21
This diff is too big to show here.
searx/translations/gl/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/gl/LC_MESSAGES/messages.po+52−51
This diff is too big to show here.
searx/translations/he/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/he/LC_MESSAGES/messages.po+18−17
This diff is too big to show here.
searx/translations/hr/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/hr/LC_MESSAGES/messages.po+23−22
This diff is too big to show here.
searx/translations/hu/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/ia/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/it/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/it/LC_MESSAGES/messages.po+23−23
This diff is too big to show here.
searx/translations/ja/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/lt/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/lt/LC_MESSAGES/messages.po+24−24
This diff is too big to show here.
searx/translations/nl/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/nl/LC_MESSAGES/messages.po+21−20
This diff is too big to show here.
searx/translations/nl_BE/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/nl_BE/LC_MESSAGES/messages.po+20−19
This diff is too big to show here.
searx/translations/oc/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/pl/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/pt/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/pt_BR/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/pt_BR/LC_MESSAGES/messages.po+20−19
This diff is too big to show here.
searx/translations/ro/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/ru/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/ru/LC_MESSAGES/messages.po+22−21
This diff is too big to show here.
searx/translations/sk/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/sl/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/sr/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/sr/LC_MESSAGES/messages.po+24−23
This diff is too big to show here.
searx/translations/sv/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/sv/LC_MESSAGES/messages.po+22−22
This diff is too big to show here.
searx/translations/ta/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/te/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/tr/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/tr/LC_MESSAGES/messages.po+135−134
This diff is too big to show here.
searx/translations/uk/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/vi/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/zh_CN/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/zh_CN/LC_MESSAGES/messages.po+21−20
This diff is too big to show here.
searx/translations/zh_Hant_TW/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/zh_TW/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/zh_TW/LC_MESSAGES/messages.po+19−18
This diff is too big to show here.
searx/utils.py+147−57
This diff is too big to show here.
searx/version.py+2−2
This diff is too big to show here.
searx/webadapter.py+34−40
This diff is too big to show here.
searx/webapp.py+277−128
This diff is too big to show here.
searx/webutils.py+4−1
This diff is too big to show here.
searx_extra/__init__.py
This diff is too big to show here.
searx_extra/google_search.py
This diff is too big to show here.
searx_extra/standalone_searx.py+217
This diff is too big to show here.
searx_extra/update/__init__.py
This diff is too big to show here.
searx_extra/update/update_ahmia_blacklist.py+1−4
This diff is too big to show here.
searx_extra/update/update_currencies.py+150
This diff is too big to show here.
searx_extra/update/update_engine_descriptions.py+202
This diff is too big to show here.
searx_extra/update/update_external_bangs.py+157
This diff is too big to show here.
searx_extra/update/update_firefox_version.py+1−6
This diff is too big to show here.
searx_extra/update/update_languages.py+6−8
This diff is too big to show here.
searx_extra/update/update_osm_keys_tags.py+209
This diff is too big to show here.
searx_extra/update/update_pygments.py+138
This diff is too big to show here.
searx_extra/update/update_translations.sh
This diff is too big to show here.
searx_extra/update/update_wikidata_units.py+54
This diff is too big to show here.
setup.py+5−6
This diff is too big to show here.
tests/test_robot.py−23
This diff is too big to show here.
tests/unit/engines/test_command.py−1
This diff is too big to show here.
tests/unit/network/__init__.py
This diff is too big to show here.
tests/unit/network/test_network.py+239
This diff is too big to show here.
tests/unit/settings/empty_settings.yml
This diff is too big to show here.
tests/unit/settings/syntaxerror_settings.yml+2
This diff is too big to show here.
tests/unit/settings/user_settings.yml+111
This diff is too big to show here.
tests/unit/settings/user_settings_keep_only.yml+14
This diff is too big to show here.
tests/unit/settings/user_settings_remove.yml+10
This diff is too big to show here.
tests/unit/settings/user_settings_remove2.yml+15
This diff is too big to show here.
tests/unit/settings/user_settings_simple.yml+9
This diff is too big to show here.
tests/unit/test_engines_init.py+1−1
This diff is too big to show here.
tests/unit/test_external_bangs.py+117
This diff is too big to show here.
tests/unit/test_plugins.py+2−2
This diff is too big to show here.
tests/unit/test_preferences.py+8−12
This diff is too big to show here.
tests/unit/test_query.py+192
This diff is too big to show here.
tests/unit/test_results.py+4−4
This diff is too big to show here.
tests/unit/test_search.py+22−9
This diff is too big to show here.
tests/unit/test_settings_loader.py+122
This diff is too big to show here.
tests/unit/test_standalone_searx.py+111
This diff is too big to show here.
tests/unit/test_utils.py+92−2
This diff is too big to show here.
tests/unit/test_webadapter.py+3−3
This diff is too big to show here.
tests/unit/test_webapp.py+78−11
This diff is too big to show here.
tests/unit/test_webutils.py+22
This diff is too big to show here.
utils/brand.env+4−1
This diff is too big to show here.
utils/build_env.py+38
This diff is too big to show here.
utils/fabfile.py−117
This diff is too big to show here.
utils/fetch_currencies.py−163
This diff is too big to show here.
utils/fetch_wikidata_units.py−47
This diff is too big to show here.
utils/filtron.sh+2−2
This diff is too big to show here.
utils/lib.sh+327−17
This diff is too big to show here.
utils/lxc-searx.env+3−4
This diff is too big to show here.
utils/lxc.sh+10−9
This diff is too big to show here.
utils/makefile.include+12−51
This diff is too big to show here.
utils/makefile.lxc+5−2
This diff is too big to show here.
utils/makefile.python−263
This diff is too big to show here.
utils/makefile.sphinx−201
This diff is too big to show here.
utils/searx.sh+27−17
This diff is too big to show here.
utils/site-python/sphinx_build_tools.py−48
This diff is too big to show here.
utils/standalone_searx.py−105
This diff is too big to show here.
utils/templates/etc/nginx/default.apps-available/morty.conf+1−1
This diff is too big to show here.
utils/templates/etc/nginx/default.apps-available/searx.conf:filtron+3−3
This diff is too big to show here.
utils/templates/etc/searx/use_default_settings.yml+22
This diff is too big to show here.
utils/templates/etc/uwsgi/apps-archlinux/searx.ini+4−1
This diff is too big to show here.
utils/templates/etc/uwsgi/apps-available/searx.ini+3
This diff is too big to show here.