728 files changed+99230−96452
.config.sh+1−5
@@ -26,12 +26,8 @@ fi
2626# ---------
2727
2828# SEARX_INTERNAL_URL="127.0.0.1:8888"
29−# SEARX_SETTINGS_TEMPLATE="${REPO_ROOT}/utils/templates/etc/searx/use_default_settings.yml"
3029
31−# Only change, if you maintain a searx brand in your searx fork (GIT_URL) which
32−# is not hold by branch 'master'. The branch has to be a local branch, in the
33−# repository from which you install (which is most often the case). If you want
34−# to install branch 'foo', don't forget to run 'git branch foo origin/foo' once.
30+# Only change, if you maintain a searx brand in your searx fork.
3531# GIT_BRANCH="${GIT_BRANCH:-master}"
3632
3733# filtron.sh
.coveragerc+10−1
@@ -1,6 +1,15 @@
11[run]
22branch = True
3−source = searx
3+source =
4+ searx/engines
5+ searx/__init__.py
6+ searx/autocomplete.py
7+ searx/https_rewrite.py
8+ searx/languages.py
9+ searx/search.py
10+ searx/testing.py
11+ searx/utils.py
12+ searx/webapp.py
413
514[report]
615show_missing = True
.dir-locals.el+1−1
@@ -25,7 +25,7 @@
2525;; Alternatively create the virtualenv, source it and install jedi + epc
2626;; (required by `emacs-jedi <https://tkf.github.io/emacs-jedi>`_)::
2727;;
28−;; $ python -m venv ./local/py3
28+;; $ virtualenv --python=python3 "--no-site-packages" ./local/py3
2929;; ...
3030;; $ source ./local/py3/bin/activate
3131;; (py3)$ # now install into the activated 'py3' environment ..
.github/ISSUE_TEMPLATE/bug-report.mddeleted−39
@@ -1,39 +0,0 @@
1−---
2−name: Bug report
3−about: Report a bug in Searx
4−title: ''
5−labels: bug
6−assignees: ''
7−
8−---
9−<!-- PLEASE FILL THESE FIELDS, IT REALLY HELPS THE MAINTAINERS OF SEARX -->
10−
11−**Version of Searx, commit number if you are using on master branch and stipulate if you forked Searx**
12−<!-- If you are running on master branch using git execute this command
13−in order to fetch the latest commit ID:
14−```
15−git log -1
16−```
17−If you are using searx-docker then look at the bottom of the Searx page
18−and check for the version after "Powered by searx"
19−
20−Please also stipulate if you are using a forked version of Searx and
21−include a link to the fork source code.
22−-->
23−**How did you install Searx?**
24−<!-- Did you install Searx using the official wiki or using searx-docker
25−or manually by executing the searx/webapp.py file? -->
26−**What happened?**
27−<!-- A clear and concise description of what the bug is. -->
28−
29−**How To Reproduce**
30−<!-- How can we reproduce this issue? (as minimally and as precisely as possible) -->
31−
32−**Expected behavior**
33−<!-- A clear and concise description of what you expected to happen. -->
34−
35−**Screenshots & Logs**
36−<!-- If applicable, add screenshots, logs to help explain your problem. -->
37−
38−**Additional context**
39−<!-- Add any other context about the problem here. -->
.github/ISSUE_TEMPLATE/config.ymldeleted−14
@@ -1,14 +0,0 @@
1−blank_issues_enabled: true
2−contact_links:
3− - name: Installation issue using the official wiki
4− url: https://github.com/searx/searx/discussions/new?category_id=32001257
5− about: Ask for help if you are having some issue with installing Searx using the official wiki.
6− - name: Installation issue using searx-docker
7− url: https://github.com/searx/searx/discussions/new?category_id=32001259
8− about: Ask for help if you are having some issue with installing Searx using searx-docker.
9− - name: Installation issue without using the official wiki nor searx-docker
10− url: https://github.com/searx/searx/discussions/new?category_id=32001260
11− about: Ask for help if you are having some issue with installing Searx manually without using the wiki nor searx-docker.
12− - name: Report a bug on a public Searx instance
13− url: https://github.com/searx/searx/discussions/new?category_id=32001412
14− about: Report a bug that you discovered on a public Searx instance.
.github/ISSUE_TEMPLATE/engine-request.mddeleted−31
@@ -1,31 +0,0 @@
1−---
2−name: Engine request
3−about: Request a new engine in Searx
4−title: ''
5−labels: enhancement, engine request
6−assignees: ''
7−
8−---
9−<!-- PLEASE FILL THESE FIELDS, IT REALLY HELPS THE MAINTAINERS OF SEARX -->
10−
11−**Working URL to the engine**
12−<!-- Please check if the engine is responding correctly before submitting it. -->
13−
14−**Why do you want to add this engine?**
15−<!-- What's special about this engine? Is it open source or libre? -->
16−
17−**Features of this engine**
18−<!-- Features of this engine: Doesn't track its users, fast, easy to integrate, ... -->
19−
20−**How can Searx fetch the information from this engine?**
21−<!-- List API URL, example code (using the correct markdown) and more
22−that could be useful for the developers in order to implement this engine.
23−If you don't know what to write, let this part blank.>
24−
25−**Applicable category of this engine**
26−<!-- Where should this new engine fit in Searx? Current categories in Searx:
27−general, files, images, it, map, music, news, science, social media and videos.
28−You can add multiple categories at the same time. -->
29−
30−**Additional context**
31−<!-- Add any other context about this engine here. -->
.github/ISSUE_TEMPLATE/feature-request.mddeleted−21
@@ -1,21 +0,0 @@
1−---
2−name: Feature request
3−about: Request a new feature in Searx
4−title: ''
5−labels: enhancement
6−assignees: ''
7−
8−---
9−<!-- PLEASE FILL THESE FIELDS, IT REALLY HELPS THE MAINTAINERS OF SEARX -->
10−
11−**Is your feature request related to a problem? Please describe.**
12−<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
13−
14−**Describe the solution you'd like**
15−<!-- A clear and concise description of what you want to happen. -->
16−
17−**Describe alternatives you've considered**
18−<!-- A clear and concise description of any alternative solutions or features you've considered. -->
19−
20−**Additional context**
21−<!-- Add any other context or screenshots about the feature request here. -->
.github/dependabot.ymldeleted−10
@@ -1,10 +0,0 @@
1−# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
2−version: 2
3−updates:
4− - package-ecosystem: "pip"
5− directory: "/"
6− schedule:
7− interval: "weekly"
8− day: "friday"
9− open-pull-requests-limit: 5
10− target-branch: "master"
.github/workflows/data-update.ymldeleted−66
@@ -1,66 +0,0 @@
1−name: "Update searx.data"
2−on:
3− schedule:
4− - cron: "05 06 1 * *"
5− workflow_dispatch:
6−
7−jobs:
8− updateData:
9− name: Update data - ${{ matrix.fetch }}
10− runs-on: ubuntu-20.04
11− if: ${{ github.repository_owner == 'searx'}}
12− strategy:
13− matrix:
14− fetch:
15− - update_ahmia_blacklist.py
16− - update_currencies.py
17− - update_external_bangs.py
18− - update_firefox_version.py
19− - update_languages.py
20− - update_wikidata_units.py
21− steps:
22− - name: Checkout
23− uses: actions/checkout@v2
24−
25− - name: Install Ubuntu packages
26− run: |
27− sudo ./utils/searx.sh install packages
28−
29− - name: Set up Python
30− uses: actions/setup-python@v2
31− with:
32− python-version: '3.10'
33− architecture: 'x64'
34−
35− - name: Install Python dependencies
36− run: |
37− make V=1 install
38−
39− - name: Fetch data
40− env:
41− FETCH_SCRIPT: ./searx_extra/update/${{ matrix.fetch }}
42− run: |
43− V=1 ./manage pyenv.cmd python "$FETCH_SCRIPT"
44−
45− - name: Create Pull Request
46− id: cpr
47− uses: peter-evans/create-pull-request@v3
48− with:
49− token: ${{ secrets.DATA_PR_TOKEN }}
50− commit-message: Update searx.data - ${{ matrix.fetch }}
51− committer: searx-bot <noreply@github.com>
52− author: ${{ github.actor }} <${{ github.actor }}@users.noreply.github.com>
53− signoff: false
54− branch: update_data_${{ matrix.fetch }}
55− delete-branch: true
56− draft: false
57− title: 'Update searx.data - ${{ matrix.fetch }}'
58− body: |
59− Update searx.data - ${{ matrix.fetch }}
60− labels: |
61− data
62−
63− - name: Check outputs
64− run: |
65− echo "Pull Request Number - ${{ steps.cpr.outputs.pull-request-number }}"
66− echo "Pull Request URL - ${{ steps.cpr.outputs.pull-request-url }}"
.github/workflows/integration.ymldeleted−151
@@ -1,151 +0,0 @@
1−name: Integration
2−
3−on:
4− push:
5− branches: ["master"]
6− pull_request:
7− branches: ["master"]
8−
9−jobs:
10− python:
11− name: Python ${{ matrix.python-version }}
12− runs-on: ubuntu-20.04
13− strategy:
14− matrix:
15− os: [ubuntu-20.04]
16− python-version: [3.7, 3.8, 3.9, "3.10"]
17− steps:
18− - name: Checkout
19− uses: actions/checkout@v2
20− - name: Install Ubuntu packages
21− run: |
22− sudo ./utils/searx.sh install packages
23− sudo apt install firefox
24− - name: Set up Python
25− uses: actions/setup-python@v2
26− with:
27− python-version: ${{ matrix.python-version }}
28− architecture: 'x64'
29− - name: Cache Python dependencies
30− id: cache-python
31− uses: actions/cache@v2
32− with:
33− path: ./local
34− key: python-${{ matrix.os }}-${{ matrix.python-version }}-${{ hashFiles('requirements*.txt', 'setup.py') }}
35− - name: Install Python dependencies
36− if: steps.cache-python.outputs.cache-hit != 'true'
37− run: |
38− make V=1 install
39− make V=1 gecko.driver
40− - name: Run tests
41− run: make V=1 ci.test
42− - name: Test coverage
43− run: make V=1 test.coverage
44− - name: Store coverage result
45− uses: actions/upload-artifact@v2
46− with:
47− name: coverage-${{ matrix.python-version }}
48− path: coverage/
49− retention-days: 60
50−
51− themes:
52− name: Themes
53− runs-on: ubuntu-20.04
54− steps:
55− - name: Checkout
56− uses: actions/checkout@v2
57− - name: Install Ubuntu packages
58− run: sudo ./utils/searx.sh install packages
59− - name: Set up Python
60− uses: actions/setup-python@v2
61− with:
62− python-version: '3.9'
63− architecture: 'x64'
64− - name: Cache Python dependencies
65− id: cache-python
66− uses: actions/cache@v2
67− with:
68− path: ./local
69− key: python-ubuntu-20.04-3.9-${{ hashFiles('requirements*.txt', 'setup.py') }}
70− - name: Install node dependencies
71− run: make V=1 node.env
72− - name: Build themes
73− run: make V=1 themes.all
74−
75− documentation:
76− name: Documentation
77− runs-on: ubuntu-20.04
78− steps:
79− - name: Checkout
80− uses: actions/checkout@v2
81− with:
82− fetch-depth: '0'
83− persist-credentials: false
84− - name: Install Ubuntu packages
85− run: sudo ./utils/searx.sh install buildhost
86− - name: Set up Python
87− uses: actions/setup-python@v2
88− with:
89− python-version: '3.10'
90− architecture: 'x64'
91− - name: Cache Python dependencies
92− id: cache-python
93− uses: actions/cache@v2
94− with:
95− path: ./local
96− key: python-ubuntu-20.04-3.9-${{ hashFiles('requirements*.txt', 'setup.py') }}
97− - name: Build documentation
98− run: |
99− make V=1 docs.clean docs.html
100− - name: Deploy
101− if: github.ref == 'refs/heads/master'
102− uses: JamesIves/github-pages-deploy-action@3.7.1
103− with:
104− GITHUB_TOKEN: ${{ github.token }}
105− BRANCH: gh-pages
106− FOLDER: dist/docs
107− CLEAN: true # Automatically remove deleted files from the deploy branch
108−
109− dockers:
110− name: Docker
111− if: github.ref == 'refs/heads/master'
112− needs:
113− - python
114− - themes
115− - documentation
116− env:
117− DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
118− runs-on: ubuntu-20.04
119− steps:
120− - name: Checkout
121− if: env.DOCKERHUB_USERNAME != null
122− uses: actions/checkout@v2
123− with:
124− # make sure "make docker.push" can get the git history
125− fetch-depth: '0'
126− - name: Set up Python
127− uses: actions/setup-python@v2
128− with:
129− python-version: '3.9'
130− architecture: 'x64'
131− - name: Cache Python dependencies
132− id: cache-python
133− uses: actions/cache@v2
134− with:
135− path: ./local
136− key: python-ubuntu-20.04-3.9-${{ hashFiles('requirements*.txt', 'setup.py') }}
137− - name: Set up QEMU
138− if: env.DOCKERHUB_USERNAME != null
139− uses: docker/setup-qemu-action@v1
140− - name: Set up Docker Buildx
141− if: env.DOCKERHUB_USERNAME != null
142− uses: docker/setup-buildx-action@v1
143− - name: Login to DockerHub
144− if: env.DOCKERHUB_USERNAME != null
145− uses: docker/login-action@v1
146− with:
147− username: ${{ secrets.DOCKERHUB_USERNAME }}
148− password: ${{ secrets.DOCKERHUB_TOKEN }}
149− - name: Build and push
150− if: env.DOCKERHUB_USERNAME != null
151− run: make -e GIT_URL=$(git remote get-url origin) docker.push
.gitignore−2
@@ -26,5 +26,3 @@ dist/
2626local/
2727gh-pages/
2828searx.egg-info/
29−.env
30−geckodriver.log
.pylintrc+2−2
@@ -12,7 +12,7 @@
1212# A comma-separated list of package or module names from where C extensions may
1313# be loaded. Extensions are loading into the active Python interpreter and may
1414# run arbitrary code
15−extension-pkg-whitelist=lxml.etree
15+extension-pkg-whitelist=
1616
1717# Add files or directories to the blacklist. They should be base names, not
1818# paths.
@@ -59,7 +59,7 @@ confidence=
5959# --enable=similarities". If you want to run only the classes checker, but have
6060# no Warning level messages displayed, use"--disable=all --enable=classes
6161# --disable=W"
62−disable=bad-whitespace, duplicate-code, consider-using-f-string
62+disable=bad-whitespace, duplicate-code
6363
6464# Enable the message, report, category or checker with the given id(s). You can
6565# either give multiple identifier separated by comma (,) or put this option
.travis.ymlnew file+77
@@ -0,0 +1,77 @@
1+os: linux
2+dist: bionic
3+language: python
4+cache:
5+ - directories:
6+ - $HOME/.cache/pip
7+addons:
8+ firefox: "latest"
9+
10+install:
11+ - env
12+ - which python; python --version
13+ - make V=1 install
14+ - make V=1 gecko.driver
15+ - make V=1 node.env
16+ - make V=1 travis.codecov
17+script:
18+ - make V=1 themes
19+ - make V=1 test
20+after_success:
21+ - make V=1 test.coverage
22+ - codecov
23+
24+stages:
25+ - test
26+ - name: documentation
27+ if: env(GITHUB_TOKEN) IS present
28+ - name: docker
29+ if: branch = master AND type != pull_request AND env(DOCKER_USERNAME) IS present
30+
31+jobs:
32+ include:
33+ - python: "3.5"
34+ - python: "3.6"
35+ - python: "3.7"
36+ - python: "3.8"
37+ - stage: documentation
38+ python: "3.8"
39+ addons: []
40+ before_install:
41+ - sudo ./utils/searx.sh install buildhost
42+ install: true
43+ script:
44+ - SEARX_DEBUG=1 make travis-gh-pages
45+ after_success: true
46+ deploy:
47+ provider: pages
48+ edge: false
49+ token: $GITHUB_TOKEN # Set in the settings page of your repository, as a secure variable
50+ strategy: git
51+ skip_cleanup: true
52+ keep_history: true
53+ local_dir: gh-pages
54+ target_branch: gh-pages
55+ on:
56+ branch: master
57+ - stage: docker
58+ python: "3.8"
59+ git:
60+ depth: false
61+ services:
62+ - docker
63+ addons: []
64+ before_install: true
65+ install: true
66+ script:
67+ - echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin
68+ - make -e GIT_URL=$(git remote get-url origin) docker.push
69+ after_success: true
70+
71+notifications:
72+ irc:
73+ channels:
74+ - "irc.freenode.org#searx"
75+ template:
76+ - "%{repository}/#%{build_number}/%{branch} (%{author}): %{message} %{build_url}"
77+ on_success: change
AUTHORS.rst+4−36
@@ -1,4 +1,4 @@
1−Searx was created by Adam Tauber and is maintained by Adam Tauber, Noémi Ványi, @pofilo, Gaspard d'Hautefeuille and Émilien Devos.
1+Searx was created by Adam Tauber and is maintained by Adam Tauber, Alexandre Flament, Noémi Ványi, @pofilo, Gaspard d'Hautefeuille and Markus Heiser.
22
33Major contributing authors:
44
@@ -11,10 +11,8 @@ Major contributing authors:
1111- Marc Abonce Seguin @a01200356
1212- @pofilo
1313- Markus Heiser @return42
14−- Émilien Devos @unixfox
15−- Alexandre Flament
1614
17−People who have submitted patches/translations, reported bugs, consulted features or
15+People who have submitted patches/translates, reported bugs, consulted features or
1816generally made searx better:
1917
2018- Laszlo Hammerl
@@ -67,6 +65,7 @@ generally made searx better:
6765- @firebovine
6866- Lorenzo J. Lucchini @luccoj
6967- @eig8phei
68+- Joachim Cherqui
7069- @maxigas
7170- Jannik Winkel @kiney
7271- @juanitobananas
@@ -111,6 +110,7 @@ generally made searx better:
111110- Nick Espig @nachtalb
112111- Rachmadani Haryono @rachmadaniHaryono
113112- Frank de Lange @yetangitu
113+- Émilien Devos @unixfox
114114- Nicolas Gelot @nfk
115115- @volth
116116- Mathieu Brunot @madmath03
@@ -134,35 +134,3 @@ generally made searx better:
134134- Sophie Tauchert @999eagle
135135- @bauruine
136136- Michael Ilsaas `<https://mikeri.net>`_
137−- @renyhp
138−- rachmadani haryono @rachmadaniHaryono
139−- Mohamad Safadieh @msafadieh
140−- @gardouille
141−- @resynth1943
142−- @Eliesemoule
143−- @gardouille
144−- @GazoilKerozen
145−- Lukáš Kucharczyk @KucharczykL
146−- Lynda Lopez @lyndalopez544
147−- M. Efe Çetin @efectn
148−- Nícholas Kegler @nicholasks
149−- @pierrechtux
150−- Scott Wallace @scottwallacesh
151−- @Singustromo
152−- @TheEvilSkeleton
153−- @Wonderfall
154−- @mrwormo
155−- Xiaoyu WEI @xywei
156−- @joshu9h
157−- Daniel Hones
158−- @cyclaero
159−- @thezeroalpha
160−- @Tobi823
161−- @archiecodes
162−- @BBaoVanC
163−- @datagram1
164−- @lucky13820
165−- @jhigginbotham
166−- @xenrox
167−- @OliveiraHermogenes
168−- Paul Alcock @Guilvareux
CHANGELOG.rst−283
@@ -1,286 +1,3 @@
1−1.0.0 2021.03.27
2−================
3−
4−Core
5−~~~~
6−
7−- drop support for Python 3.5 ( #2459 )
8−- add support for Python 3.9 ( #2397 #2459 )
9−- update Python dependencies ( #2428 #2459 #2206 ) ⚠️ pyopenssl is not longer required
10−- automatic update of searx.data ( #2555 #2585 #2595 #2592 #2600 )
11−- update searx.data ( #2604 #2605 #2606 #2607 #2415 )
12−- add ability to send engine data to subsequent requests ( #2615 )
13−- add checker ( #2419 #2476 #2481 #2682 #2682 #2657 )
14−- by default allow only HTTPS, not HTTP ( #2641 #2659 )
15−- replace /translations.js with an embedded JSON ( #2660 )
16−- activate raise_for_error by default ( #2557 )
17−- don't dump traceback of SearxEngineResponseException on init ( #2635 )
18−
19−Documentation
20−~~~~~~~~~~~~~
21−
22−- update nginx configuration ( #2618 )
23−- document workaround for using 2 languages simultaneously ( #2479 )
24−- improve admin-docs about result proxy (morty) configuration ( #2509 )
25−- fixed typo ( #2457 )
26−
27−New settings.yml
28−~~~~~~~~~~~~~~~~
29−
30−- `general.contact_url` : add link to contact instance maintainer to footer of each page ( #2391 14c7cc0e118f1d0873b32b34793cdec2c5c9c13e #2412 )
31−- `brand` : move brand options from Makefile to settings.yml ( #2408 #2473 )
32−
33−Themes
34−~~~~~~
35−
36−- oscar: Hide links panel in mobile screens ( #2458 )
37−- oscar: upgrade dependencies ( #2346 #2673 #2662 )
38−- remove legacy, courgette and pix-art themes ( #2344 )
39−- add hyperlink to searx instances list in error message ( #2387 )
40−- preferences: a tooltip is shown when the mouse is over the engine names ( #2661 )
41−- Ignore double-quotes when highlighting query parts ( #2553 )
42−- update autocomplete ( #2593 )
43−
44−New engines
45−~~~~~~~~~~~
46−
47−- ccengine ( #2533 )
48−- mediathekviewweb ( #2541 )
49−- solidtorrents ( #2626 )
50−- solr ( #2652 )
51−- rumble ( #2588 )
52−- wiby.me ( #2452 )
53−
54−Fixed engines
55−~~~~~~~~~~~~~
56−
57−- apk_mirror ( #2556 #2642 )
58−- bing ( #2602 )
59−- duckduckgo ( #2560 #2559 )
60−- library genesis ( #2448 )
61−- ina ( 0ba71c3644c4d20f70528c10eed1385399ec1c82 )
62−- invidious ( #2451 )
63−- json_engine ( #2562 )
64−- google ( #2482 )
65−- google images ( #2482 )
66−- google play apps ( 88657fe9c2a41b9be38ee5146e5870672416db12 )
67−- google play movies ( 50ba2b9e87ef61e96da124f906d3aff4c7870e3f )
68−- google news ( #2483 #2498 )
69−- google scholar ( #2611 )
70−- google video ( #2482 )
71−- hoogle ( 6255b33c9dcf0d28f0a3307af988565f69259ce2 )
72−- naver ( #2542 )
73−- semantic schollar ( f596f5767bed915a5c3bed59ae26283e53f975ca f596f5767bed915a5c3bed59ae26283e53f975ca )
74−- startpage ( #2396 )
75−- seznam ( #2564 28286cf3f2308113bf440fb6e7cf326c6ed07889 )
76−- wikipedia ( #2554 #2565 #2681 #2681 )
77−- yacy ( #2669 )
78−- yahoo news ( #2640 #2655 )
79−
80−Updated engines
81−~~~~~~~~~~~~~~~
82−
83−- duckduckgo ( 5f450fda74e80bf350eb1493f66cfa61deaf5cea )
84−- geektimes ( 45f0e1a859fa12ce2ae0c24dc356922fcad50c8d )
85−- lobste.rs ( 06b754ad67aa6066aed6df77b5ffb74aabebb040 )
86−- soundcloud ( #2671 )
87−- peertube ( #2570 )
88−- recoll ( #2539 )
89−- yggtorrent ( #2573 )
90−
91−Removed engines
92−~~~~~~~~~~~~~~~
93−
94−- acgsou ( #2654 )
95−- google_play_music ( #2558 )
96−- metager ( #2538 )
97−- voat ( #2445 )
98−- yandex ( #2566 )
99−
100−Bug fixes
101−~~~~~~~~~
102−
103−- Fix empty colon in query from selecting Chinese ( #2454 )
104−- Get correct locale with country from browser ( #2531 )
105−
106−Code refactoring / reduce the technical debt
107−~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
108−
109−- refactor searx.search.SearchQuery and searx.search.EngineRef ( #2398 )
110−- dynamically set language_support variable ( #2499 )
111−- engines: add about variable ( #2460 )
112−- processors ( #2225 5c6a5407a0b124c3323e73c33b81ec1fbd7d2fce )
113−- remove Fabric file ( #2494 )
114−- use unittest from py3, remove unittest2 from py2 ( #2608 )
115−
116−Github
117−~~~~~~
118−
119−- add notice for the issue templates ( #2447 )
120−- every Sunday, call utils/fetch_*.py scripts and create a PR automatically ( #2500 728e09676400221a064627509a31470d8f6e33bf )
121−- minor change: replace "travis" by "CI" ( #2528 )
122−
123−Build scripts
124−~~~~~~~~~~~~~
125−
126−- update secret key check ( #2411 )
127−- fix makefile targets `books/{name}.*` and `books/user.pdf` ( #2420 #2530 )
128−- upload-pypi-test & linuxdoc has been released on PyPi ( #2456 )
129−- fix makefile target `gh-pages` : flatten history of branch gh.pages ( #2514 )
130−- optimize creation of the virtualenv & pyenvinstall targets ( #2421 )
131−- update pyenv pyenvinstall Make targets ( #2517 )
132−- makefile.python: remove duplicate pyenv-(un)install targets ( #2418 )
133−- [fix] make targets engines.languages and useragents.update ( #2643 )
134−- [fix] utils/serax.sh create_pyenv() - drop duplicate 'pip install .' ( #2621 )
135−
136−Install scripts
137−~~~~~~~~~~~~~~~
138−
139−- drop Ubuntu 16.04 (Xenial Xerus) support ( #2619 )
140−- replace ubu1910 image by ubu2010 image ( #2435 )
141−- LXC switch to Fedora 33 / Fedora 31 reached its EOL #2634 ( #2634 )
142−- add package which to CentOS-7 boilerplate ( #2623 )
143−- use SEARX_SETTINGS_TEMPLATE from .config environment ( #2417 )
144−- determine path to makefile.lxc in a LXC ( #2399 )
145−- remove unused code ( #2401 #2497 )
146−- support git versions <v2.22 ( #2620 )
147−
148−Announcement
149−~~~~~~~~~~~~
150−
151−We, the searx maintainer team, would like to say a huge thank you for everybody who had been involved in the development of searx or supported us in the past 7 years - making our first stable release available. Special thanks to [NLNet](https://nlnet.nl) for sponsoring multiple features of this release.
152−
153−
154−0.18.0 2020.12.14
155−=================
156−
157−Core
158−~~~~
159−
160−- drop Python 2 support ( #2137 #2244 )
161−- separate index and search routes ( #1681 ). ⚠️ add & remove your searx instance(s) from your browser.
162−- add external_bang ( #2027 #2043 #2059 )
163−- add external plugins supports ( #2074 )
164−- add plugin converting strings into hash digests ( #1246 )
165−- new category: Onions ( #565 )
166−- allow searx query parts anywhere in the query ( commit aa3c18dda9329fff875328f6ba97483c417b149a 2aef38c3b9d1fe93e9d665a49b10151d63d92392 )
167−- preferences: use base_url for prefix of sharing 'currenly saved preferences' (#1249 )
168−- upgrade to request 2.24.0, pyopenssl is optional ( #2199 )
169−- force admins to set secret_key if debug mode is disabled ( #2256 )
170−- standalone searx update ( #1591 )
171−- architecture clean up ( #2140 #2185 #2195 #2196 #2198 #2189 #2208 #2239 #2241 #2246 #2248 )
172−- record detail about engine error ( #2332 #2375 #2350 ). Add a new API endpoint: ``/stats/errors``.
173−- display if an engine does not support HTTPS requests ( #2373 )
174−
175−New settings.yml
176−~~~~~~~~~~~~~~~~
177−
178−- ``use_default_settings``: user settings can relied on the default settings ( #2291 #2362 #2349 )
179−- ``ui.results_on_new_tab: False`` - for opening result links in a new tab ( #2167 )
180−- ``ui.advanced_search`` - add preference for displaying advanced settings ( #2327 )
181−- ``server.method: "POST"`` - Make default query submission method configurable ( #2130 )
182−- ``server.default_http_headers`` - add default http headers ( #2295 )
183−- ``engines.*.proxies`` - Using proxy only for specific engines ( #1827 #2319 ), see https://searx.github.io/searx/dev/engine_overview.html#settings-yml
184−- ``enabled_plugins`` - Enabled plugins ( a05c660e3036ad8d02072fc6731af54c2ed6151c )
185−- ``preferences.lock`` - Let admins lock user preferences ( #2270 )
186−
187−Oscar theme
188−~~~~~~~~~~~
189−
190−- update infobox ( #2131 )
191−
192− - Make infoboxes shorter by default.
193− - Hide the main image by default as well and set a maximum height even when expanded.
194− - Add a toggle at the bottom of the infobox to expand it or to shrink it again.
195− - Fix pointhi style
196−- query suggestion does not keep the language tag of the original query ( #1314 )
197−- fix the clear button ( #2306 )
198−
199−Simple theme
200−~~~~~~~~~~~~
201−
202−- Fix autocomplete ( #2205 )
203−
204−New engines
205−~~~~~~~~~~~
206−
207−- ahmia, not_evil ( #565 )
208−- codeberg ( #2104 )
209−- command line engines: git grep, find, etc. ( #2128 #2250 )
210−- elasticsearch ( #2292 )
211−- metager ( #2139 )
212−- naver ( #1912 )
213−- opensemanticsearch ( #2271 )
214−- peertube ( #2109 )
215−- recoll (#2325)
216−- sepiasearch ( #2227 )
217−
218−Updated engines
219−~~~~~~~~~~~~~~~
220−
221−- digg ( #2285 )
222−- dbpedia ( #2352 )
223−- duckduckgo_definitions ( #2224 #2356 )
224−- duden ( #2359 )
225−- invidious ( #2116 )
226−- libgen ( #2360 )
227−- photon ( #2336 )
228−- soundclound ( #2365 )
229−- wikipedia ( #2178 #2363 #2354 )
230−- wikidata ( #2151 #2224 #2353 ) - faster response time
231−- yaCy ( #2255 ) - support HTTP digest authentication.
232−- youtube_noapi ( #2364 )
233−
234−Fixed engines
235−~~~~~~~~~~~~~
236−
237−- 1x ( #2361 )
238−- answer 'random sha256' ( #2121 )
239−- bing image ( #1496 )
240−- duckduckgo ( #2254 )
241−- genius ( #2371 )
242−- google ( #2236 )
243−- google image ( #2115 )
244−- lobste.rs ( #2253 )
245−- piratebay ( #2133 )
246−- startpage ( #2385 )
247−- torrentz ( #2101 )
248−
249−Removed engines
250−~~~~~~~~~~~~~~~
251−
252−- filecrop ( #2352 )
253−- searchcode_doc ( #2372 )
254−- seedpeer ( #2366 )
255−- twitter ( #2372 )
256−- yggtorrent ( #2099 #2375 )
257−
258−Install scripts & documentation
259−~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
260−
261−- install script & documentation ( #2384 #2380 #2362 #2287 #2283 #2277 #2223 #2211 #2118 #2117 #2063 )
262−
263−Docker image
264−~~~~~~~~~~~~
265−
266−- use Alpine 3.12 ( #1983 )
267−- uwsgi serves the static files directly. ( #1865 )
268−- fix k8s support ( #2099 )
269−- make docker produces clean tag version ( #2182 )
270−
271−Bug fixes
272−~~~~~~~~~
273−
274−- searx.utils.HTMLTextExtractor: invalid HTML don't raise an Exception ( #2190 )
275−- Fix static URL ( commit da8b227044f45127f705f6ea94a72d368eea73bb )
276−- Fix autocomplete ( #2127 )
277−- Fix opensearch.xml ( #2132 #2247 )
278−- Fix documentation build ( #2237 )
279−- Some fixes in the fetch languages script ( #2212 )
280−
281−Special thanks to `NLNet <https://nlnet.nl>`__ for sponsoring multiple features of this release.
282−
283−
28410.17.0 2020.07.09
2852=================
2863
CONTRIBUTING.md+2−2
@@ -2,8 +2,8 @@
22
33## Resources in the documentation
44
5−* [Development quickstart](https://searx.github.io/searx/dev/contribution_guide.html)
6−* [Contribution guide](https://searx.github.io/searx/dev/contribution_guide.html)
5+* [Development quickstart](http://searx.github.io/searx/dev/contribution_guide.html)
6+* [Contribution guide](http://searx.github.io/searx/dev/contribution_guide.html)
77
88## Submitting PRs
99
Dockerfile+15−17
@@ -1,22 +1,29 @@
1−FROM alpine:3.15
1+FROM alpine:3.12
22ENTRYPOINT ["/sbin/tini","--","/usr/local/searx/dockerfiles/docker-entrypoint.sh"]
33EXPOSE 8080
44VOLUME /etc/searx
55VOLUME /var/log/uwsgi
66
7+ARG GIT_URL=unknown
8+ARG VERSION_GITCOMMIT=unknown
9+ARG SEARX_GIT_VERSION=unknown
10+
711ARG SEARX_GID=977
812ARG SEARX_UID=977
913
1014RUN addgroup -g ${SEARX_GID} searx && \
1115 adduser -u ${SEARX_UID} -D -h /usr/local/searx -s /bin/sh -G searx searx
1216
17+ARG TIMESTAMP_SETTINGS=0
18+ARG TIMESTAMP_UWSGI=0
19+ARG LABEL_VCS_REF=
20+ARG LABEL_VCS_URL=
21+
1322ENV INSTANCE_NAME=searx \
1423 AUTOCOMPLETE= \
1524 BASE_URL= \
1625 MORTY_KEY= \
17− MORTY_URL= \
18− SEARX_SETTINGS_PATH=/etc/searx/settings.yml \
19− UWSGI_SETTINGS_PATH=/etc/searx/uwsgi.ini
26+ MORTY_URL=
2027
2128WORKDIR /usr/local/searx
2229
@@ -46,19 +53,14 @@ RUN apk upgrade --no-cache \
4653 uwsgi \
4754 uwsgi-python3 \
4855 brotli \
49− && pip3 install --upgrade pip wheel setuptools \
56+ && pip3 install --upgrade pip \
5057 && pip3 install --no-cache -r requirements.txt \
5158 && apk del build-dependencies \
5259 && rm -rf /root/.cache
5360
54−COPY searx ./searx
55−COPY dockerfiles ./dockerfiles
61+COPY --chown=searx:searx . .
5662
57−ARG TIMESTAMP_SETTINGS=0
58−ARG TIMESTAMP_UWSGI=0
59−ARG VERSION_GITCOMMIT=unknown
60−
61−RUN /usr/bin/python3 -m compileall -q searx; \
63+RUN su searx -c "/usr/bin/python3 -m compileall -q searx"; \
6264 touch -c --date=@${TIMESTAMP_SETTINGS} searx/settings.yml; \
6365 touch -c --date=@${TIMESTAMP_UWSGI} dockerfiles/uwsgi.ini; \
6466 if [ ! -z $VERSION_GITCOMMIT ]; then\
@@ -68,12 +70,8 @@ RUN /usr/bin/python3 -m compileall -q searx; \
6870 -o -name '*.svg' -o -name '*.ttf' -o -name '*.eot' \) \
6971 -type f -exec gzip -9 -k {} \+ -exec brotli --best {} \+
7072
71−# Keep these arguments at the end to prevent redundant layer rebuilds
73+# Keep this argument at the end since it change each time
7274ARG LABEL_DATE=
73−ARG GIT_URL=unknown
74−ARG SEARX_GIT_VERSION=unknown
75−ARG LABEL_VCS_REF=
76−ARG LABEL_VCS_URL=
7775LABEL maintainer="searx <${GIT_URL}>" \
7876 description="A privacy-respecting, hackable metasearch engine." \
7977 version="${SEARX_GIT_VERSION}" \
Makefile+239−78
@@ -1,107 +1,268 @@
11# -*- coding: utf-8; mode: makefile-gmake -*-
2−# SPDX-License-Identifier: AGPL-3.0-or-later
3−
42.DEFAULT_GOAL=help
5−export MTOOLS=./manage
3+
4+# START Makefile setup
5+export GIT_URL=https://github.com/searx/searx
6+export GIT_BRANCH=master
7+export SEARX_URL=https://searx.me
8+export DOCS_URL=https://searx.github.io/searx
9+# END Makefile setup
610
711include utils/makefile.include
812
13+PYOBJECTS = searx
14+DOC = docs
15+PY_SETUP_EXTRAS ?= \[test\]
16+
17+include utils/makefile.python
18+include utils/makefile.sphinx
19+
920all: clean install
1021
11−PHONY += help
22+PHONY += help-min help-all help
23+
24+help: help-min
25+ @echo ''
26+ @echo 'to get more help: make help-all'
27+
28+help-min:
29+ @echo ' test - run developer tests'
30+ @echo ' docs - build documentation'
31+ @echo ' docs-live - autobuild HTML documentation while editing'
32+ @echo ' run - run developer instance'
33+ @echo ' install - developer install (./local)'
34+ @echo ' uninstall - uninstall (./local)'
35+ @echo ' gh-pages - build docs & deploy on gh-pages branch'
36+ @echo ' clean - drop builds and environments'
37+ @echo ' project - re-build generic files of the searx project'
38+ @echo ' buildenv - re-build environment files (aka brand)'
39+ @echo ' themes - re-build build the source of the themes'
40+ @echo ' docker - build Docker image'
41+ @echo ' node.env - download & install npm dependencies locally'
42+ @echo ''
43+ @echo 'environment'
44+ @echo ' SEARX_URL = $(SEARX_URL)'
45+ @echo ' GIT_URL = $(GIT_URL)'
46+ @echo ' DOCS_URL = $(DOCS_URL)'
47+ @echo ''
48+ @$(MAKE) -e -s make-help
1249
13−help:
14− @./manage --help
15− @echo '----'
16− @echo 'run - run developer instance'
17− @echo 'install - developer install of searx into virtualenv'
18− @echo 'uninstall - uninstall developer installation'
19− @echo 'clean - clean up working tree'
20− @echo 'search.checker - check search engines'
21− @echo 'test - run shell & CI tests'
22− @echo 'test.sh - test shell scripts'
23− @echo 'ci.test - run CI tests'
50+help-all: help-min
51+ @echo ''
52+ @$(MAKE) -e -s python-help
53+ @echo ''
54+ @$(MAKE) -e -s docs-help
2455
56+PHONY += install
57+install: buildenv pyenvinstall
58+
59+PHONY += uninstall
60+uninstall: pyenvuninstall
61+
62+PHONY += clean
63+clean: pyclean docs-clean node.clean test.clean
64+ $(call cmd,common_clean)
2565
2666PHONY += run
27−run: install
67+run: buildenv pyenvinstall
2868 $(Q) ( \
2969 sleep 2 ; \
3070 xdg-open http://127.0.0.1:8888/ ; \
3171 ) &
32− SEARX_DEBUG=1 ./manage pyenv.cmd python ./searx/webapp.py
72+ SEARX_DEBUG=1 $(PY_ENV)/bin/python ./searx/webapp.py
3373
34−PHONY += install uninstall
35−install uninstall:
36− $(Q)./manage pyenv.$@
74+# docs
75+# ----
76+
77+sphinx-doc-prebuilds:: buildenv pyenvinstall prebuild-includes
78+
79+PHONY += docs
80+docs: sphinx-doc-prebuilds
81+ $(call cmd,sphinx,html,docs,docs)
82+
83+PHONY += docs-live
84+docs-live: sphinx-doc-prebuilds
85+ $(call cmd,sphinx_autobuild,html,docs,docs)
86+
87+PHONY += prebuild-includes
88+prebuild-includes:
89+ $(Q)mkdir -p $(DOCS_BUILD)/includes
90+ $(Q)./utils/searx.sh doc | cat > $(DOCS_BUILD)/includes/searx.rst
91+ $(Q)./utils/filtron.sh doc | cat > $(DOCS_BUILD)/includes/filtron.rst
92+ $(Q)./utils/morty.sh doc | cat > $(DOCS_BUILD)/includes/morty.rst
93+
94+
95+$(GH_PAGES)::
96+ @echo "doc available at --> $(DOCS_URL)"
97+
98+# update project files
99+# --------------------
100+
101+PHONY += project engines.languages useragents.update buildenv
102+
103+project: buildenv useragents.update engines.languages
104+
105+engines.languages: pyenvinstall
106+ $(Q)echo "fetch languages .."
107+ $(Q)$(PY_ENV_ACT); python utils/fetch_languages.py
108+ $(Q)echo "update searx/data/engines_languages.json"
109+ $(Q)mv engines_languages.json searx/data/engines_languages.json
110+ $(Q)echo "update searx/languages.py"
111+ $(Q)mv languages.py searx/languages.py
112+
113+useragents.update: pyenvinstall
114+ $(Q)echo "Update searx/data/useragents.json with the most recent versions of Firefox."
115+ $(Q)$(PY_ENV_ACT); python utils/fetch_firefox_version.py
116+
117+buildenv:
118+ $(Q)echo "build searx/brand.py"
119+ $(Q)echo "GIT_URL = '$(GIT_URL)'" > searx/brand.py
120+ $(Q)echo "GIT_BRANCH = '$(GIT_BRANCH)'" >> searx/brand.py
121+ $(Q)echo "ISSUE_URL = 'https://github.com/searx/searx/issues'" >> searx/brand.py
122+ $(Q)echo "SEARX_URL = '$(SEARX_URL)'" >> searx/brand.py
123+ $(Q)echo "DOCS_URL = '$(DOCS_URL)'" >> searx/brand.py
124+ $(Q)echo "PUBLIC_INSTANCES = 'https://searx.space'" >> searx/brand.py
125+ $(Q)echo "build utils/brand.env"
126+ $(Q)echo "export GIT_URL='$(GIT_URL)'" > utils/brand.env
127+ $(Q)echo "export GIT_BRANCH='$(GIT_BRANCH)'" >> utils/brand.env
128+ $(Q)echo "export ISSUE_URL='https://github.com/searx/searx/issues'" >> utils/brand.env
129+ $(Q)echo "export SEARX_URL='$(SEARX_URL)'" >> utils/brand.env
130+ $(Q)echo "export DOCS_URL='$(DOCS_URL)'" >> utils/brand.env
131+ $(Q)echo "export PUBLIC_INSTANCES='https://searx.space'" >> utils/brand.env
37132
38−PHONY += clean
39−clean: py.clean docs.clean node.clean test.clean
40− $(Q)./manage build_msg CLEAN "common files"
41− $(Q)find . -name '*.orig' -exec rm -f {} +
42− $(Q)find . -name '*.rej' -exec rm -f {} +
43− $(Q)find . -name '*~' -exec rm -f {} +
44− $(Q)find . -name '*.bak' -exec rm -f {} +
45−
46−PHONY += search.checker search.checker.%
47−search.checker: install
48− $(Q)./manage pyenv.cmd searx-checker -v
49−
50−search.checker.%: install
51− $(Q)./manage pyenv.cmd searx-checker -v "$(subst _, ,$(patsubst search.checker.%,%,$@))"
52−
53−PHONY += ci.test test test.sh
54−ci.test: test.pep8 test.pylint test.unit test.robot
55−test: ci.test
56−test.sh:
57− $(Q)shellcheck -x -s bash \
58− utils/brand.env \
59− ./manage \
60− utils/lib.sh \
61− utils/filtron.sh \
62− utils/searx.sh \
63− utils/morty.sh \
64− utils/lxc.sh \
65− utils/lxc-searx.env \
66− .config.sh
67− $(Q)./manage build_msg TEST "$@ OK"
68133
134+# node / npm
135+# ----------
69136
70−# wrap ./manage script
137+node.env: buildenv
138+ $(Q)./manage.sh npm_packages
71139
72−MANAGE += buildenv
73−MANAGE += babel.compile
74−MANAGE += data.all data.languages data.useragents
75−MANAGE += docs.html docs.live docs.gh-pages docs.prebuild docs.clean
76−MANAGE += docker.build docker.push
77−MANAGE += gecko.driver
78−MANAGE += node.env node.clean
79−MANAGE += py.build py.clean
80−MANAGE += pyenv pyenv.install pyenv.uninstall
81−MANAGE += pypi.upload pypi.upload.test
82−MANAGE += test.pylint test.pep8 test.unit test.coverage test.robot test.clean
83−MANAGE += themes.all themes.oscar themes.simple themes.bootstrap
140+node.clean:
141+ $(Q)echo "CLEAN locally installed npm dependencies"
142+ $(Q)rm -rf \
143+ ./node_modules \
144+ ./package-lock.json \
145+ ./searx/static/themes/oscar/package-lock.json \
146+ ./searx/static/themes/oscar/node_modules \
147+ ./searx/static/themes/simple/package-lock.json \
148+ ./searx/static/themes/simple/node_modules
149+
150+# build themes
151+# ------------
152+
153+PHONY += themes.bootstrap themes themes.oscar themes.simple themes.legacy themes.courgette themes.pixart
154+themes: buildenv themes.bootstrap themes.oscar themes.simple themes.legacy themes.courgette themes.pixart
155+
156+quiet_cmd_lessc = LESSC $3
157+ cmd_lessc = PATH="$$(npm bin):$$PATH" \
158+ lessc --clean-css="--s1 --advanced --compatibility=ie9" "searx/static/$2" "searx/static/$3"
159+
160+quiet_cmd_grunt = GRUNT $2
161+ cmd_grunt = PATH="$$(npm bin):$$PATH" \
162+ grunt --gruntfile "$2"
163+
164+themes.oscar: node.env
165+ $(Q)echo '[!] build oscar theme'
166+ $(call cmd,grunt,searx/static/themes/oscar/gruntfile.js)
167+
168+themes.simple: node.env
169+ $(Q)echo '[!] build simple theme'
170+ $(call cmd,grunt,searx/static/themes/simple/gruntfile.js)
171+
172+themes.legacy: node.env
173+ $(Q)echo '[!] build legacy theme'
174+ $(call cmd,lessc,themes/legacy/less/style-rtl.less,themes/legacy/css/style-rtl.css)
175+ $(call cmd,lessc,themes/legacy/less/style.less,themes/legacy/css/style.css)
176+
177+themes.courgette: node.env
178+ $(Q)echo '[!] build courgette theme'
179+ $(call cmd,lessc,themes/courgette/less/style.less,themes/courgette/css/style.css)
180+ $(call cmd,lessc,themes/courgette/less/style-rtl.less,themes/courgette/css/style-rtl.css)
181+
182+themes.pixart: node.env
183+ $(Q)echo '[!] build pixart theme'
184+ $(call cmd,lessc,themes/pix-art/less/style.less,themes/pix-art/css/style.css)
185+
186+themes.bootstrap: node.env
187+ $(call cmd,lessc,less/bootstrap/bootstrap.less,css/bootstrap.min.css)
188+
189+
190+# docker
191+# ------
192+
193+PHONY += docker
194+docker: buildenv
195+ $(Q)./manage.sh docker_build
196+
197+docker.push: buildenv
198+ $(Q)./manage.sh docker_build push
199+
200+# gecko
201+# -----
202+
203+PHONY += gecko.driver
204+gecko.driver:
205+ $(PY_ENV_ACT); ./manage.sh install_geckodriver
206+
207+# test
208+# ----
209+
210+PHONY += test test.sh test.pylint test.pep8 test.unit test.coverage test.robot
211+test: buildenv test.pylint test.pep8 test.unit gecko.driver test.robot
212+
213+# TODO: balance linting with pylint
214+
215+test.pylint: pyenvinstall
216+ $(call cmd,pylint,\
217+ searx/preferences.py \
218+ searx/testing.py \
219+ searx/engines/gigablast.py \
220+ )
221+
222+# ignored rules:
223+# E402 module level import not at top of file
224+# W503 line break before binary operator
225+
226+# ubu1604: uses shellcheck v0.3.7 (from 04/2015), no longer supported!
227+test.sh:
228+ shellcheck -x -s bash utils/brand.env
229+ shellcheck -x utils/lib.sh
230+ shellcheck -x utils/filtron.sh
231+ shellcheck -x utils/searx.sh
232+ shellcheck -x utils/morty.sh
233+ shellcheck -x utils/lxc.sh
234+ shellcheck -x utils/lxc-searx.env
235+ shellcheck -x .config.sh
84236
85−PHONY += $(MANAGE)
237+test.pep8: pyenvinstall
238+ @echo "TEST pycodestyle (formerly pep8)"
239+ $(Q)$(PY_ENV_ACT); pycodestyle --exclude='searx/static, searx/languages.py, searx/engines/gigablast.py' \
240+ --max-line-length=120 --ignore "E402,W503" searx tests
86241
87−$(MANAGE):
88− $(Q)$(MTOOLS) $@
242+test.unit: pyenvinstall
243+ @echo "TEST tests/unit"
244+ $(Q)$(PY_ENV_ACT); python -m nose2 -s tests/unit
89245
90−# deprecated
246+test.coverage: pyenvinstall
247+ @echo "TEST unit test coverage"
248+ $(Q)$(PY_ENV_ACT); \
249+ python -m nose2 -C --log-capture --with-coverage --coverage searx -s tests/unit \
250+ && coverage report \
251+ && coverage html \
91252
92−PHONY += docs docs-clean docs-live docker themes
253+test.robot: pyenvinstall gecko.driver
254+ @echo "TEST robot"
255+ $(Q)$(PY_ENV_ACT); PYTHONPATH=. python searx/testing.py robot
93256
94−docs: docs.html
95− $(Q)./manage build_msg WARN $@ is deprecated use docs.html
257+test.clean:
258+ @echo "CLEAN intermediate test stuff"
259+ $(Q)rm -rf geckodriver.log .coverage coverage/
96260
97−docs-clean: docs.clean
98− $(Q)./manage build_msg WARN $@ is deprecated use docs.clean
99261
100−docs-live: docs.live
101− $(Q)./manage build_msg WARN $@ is deprecated use docs.live
262+# travis
263+# ------
102264
103−docker: docker.build
104− $(Q)./manage build_msg WARN $@ is deprecated use docker.build
265+travis.codecov:
266+ $(Q)$(PY_ENV_BIN)/python -m pip install codecov
105267
106−themes: themes.all
107− $(Q)./manage build_msg WARN $@ is deprecated use themes.all
268+.PHONY: $(PHONY)
README.rst+2−2
@@ -17,7 +17,7 @@
1717|OpenCollective searx backers|
1818|OpenCollective searx sponsors|
1919
20−Privacy-respecting, hackable `metasearch engine`_ / *pronunciation* **sɜːks**.
20+Privacy-respecting, hackable `metasearch engine`_ / *pronunciation* **səːks**.
2121
2222.. _metasearch engine: https://en.wikipedia.org/wiki/Metasearch_engine
2323
@@ -61,7 +61,7 @@ our homepage_.
6161.. _homepage: https://searx.github.io/searx
6262
6363contact:
64− openhub_ // twitter_ // IRC: #searx @ Libera (irc.libera.chat)
64+ openhub_ // twitter_ // IRC: #searx @ freenode
6565
6666.. _openhub: https://www.openhub.net/p/searx
6767.. _twitter: https://twitter.com/Searx_engine
dockerfiles/docker-entrypoint.sh+4−1
@@ -24,6 +24,9 @@ if [ -z "${BIND_ADDRESS}" ]; then
2424 export BIND_ADDRESS="${DEFAULT_BIND_ADDRESS}"
2525fi
2626
27+export UWSGI_SETTINGS_PATH=/etc/searx/uwsgi.ini
28+export SEARX_SETTINGS_PATH=/etc/searx/settings.yml
29+
2730# Parse special command line
2831# see docs/admin/installation-docker.rst
2932# display the help message without the version
@@ -63,7 +66,7 @@ patch_searx_settings() {
6366 CONF="$1"
6467
6568 # Make sure that there is trailing slash at the end of BASE_URL
66− # see https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Shell-Parameter-Expansion
69+ # see http://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Shell-Parameter-Expansion
6770 export BASE_URL="${BASE_URL%/}/"
6871
6972 # update settings.yml
dockerfiles/uwsgi.ini−3
@@ -42,6 +42,3 @@ static-map = /static=/usr/local/searx/searx/static
4242static-expires = /* 864000
4343static-gzip-all = True
4444offload-threads = %k
45−
46−# Cache
47−cache2 = name=searxcache,items=2000,blocks=2000,blocksize=4096,bitmap=1
docs/_themes/searx/static/searx.css+29
@@ -138,3 +138,32 @@ caption {
138138 caption-side: top;
139139 text-align: left;
140140}
141+
142+/* bugs since sphinx 3.1
143+
144+See sphinx-doc project, PR 7838 & 7484 with elementary patch to the basic CSS:
145+
146+- https://github.com/sphinx-doc/sphinx/issues/7838#issuecomment-646009605
147+- https://github.com/sphinx-doc/sphinx/pull/7484#issuecomment-646058972
148+
149+*/
150+
151+li > p:first-child {
152+ margin-top: 0;
153+}
154+
155+li > p:last-child {
156+ margin-bottom: 0;
157+}
158+
159+div.admonition dl {
160+ margin-bottom: 0;
161+}
162+
163+div.sidebar {
164+ clear: none;
165+}
166+
167+div.admonition, div.topic, pre {
168+ clear: none;
169+}
docs/admin/buildhosts.rst+5−5
@@ -49,9 +49,9 @@ Build docs
4949 - dvisvgm_
5050
5151Most of the sphinx requirements are installed from :origin:`setup.py` and the
52−docs can be build from scratch with ``make docs.html``. For better math and
53−image processing additional packages are needed. The XeTeX_ needed not only for
54−PDF creation, its also needed for :ref:`math` when HTML output is build.
52+docs can be build from scratch with ``make docs``. For better math and image
53+processing additional packages are needed. The XeTeX_ needed not only for PDF
54+creation, its also needed for :ref:`math` when HTML output is build.
5555
5656To be able to do :ref:`sphinx:math-support` without CDNs, the math are rendered
5757as images (``sphinx.ext.imgmath`` extension).
@@ -64,10 +64,10 @@ to ``imgmath``:
6464 :start-after: # sphinx.ext.imgmath setup
6565 :end-before: # sphinx.ext.imgmath setup END
6666
67−If your docs build (``make docs.html``) shows warnings like this::
67+If your docs build (``make docs``) shows warnings like this::
6868
6969 WARNING: dot(1) not found, for better output quality install \
70− graphviz from https://www.graphviz.org
70+ graphviz from http://www.graphviz.org
7171 ..
7272 WARNING: LaTeX command 'latex' cannot be run (needed for math \
7373 display), check the imgmath_latex setting
docs/admin/engines.rst+13−26
@@ -1,28 +1,14 @@
1+.. _engines generic:
2+
13=======
24Engines
35=======
46
5−Special Engine Settings
6−=======================
7−
87.. sidebar:: Further reading ..
98
109 - :ref:`settings engine`
11− - :ref:`engine settings` & :ref:`engine file`
12−
13−.. toctree::
14− :maxdepth: 1
15−
16− engines/recoll.rst
17−
18−
19−.. _engines generic:
20−
21−General Engine Settings
22−=======================
23−
24−Explanation of the :ref:`general engine configuration` shown in the table
25−:ref:`configured engines`.
10+ - :ref:`engine settings`
11+ - :ref:`engine file`
2612
2713============= =========== ==================== ============
2814:ref:`engine settings` :ref:`engine file`
@@ -33,7 +19,7 @@ Engine .. Paging support **P**
3319------------------------- -------------------- ------------
3420Shortcut **S** Language support **L**
3521Timeout **TO** Time range support **TR**
36−Disabled **D** Engine type **ET**
22+Disabled **D** Offline **O**
3723------------- ----------- -------------------- ------------
3824Safe search **SS**
3925------------- ----------- ---------------------------------
@@ -44,9 +30,11 @@ Disabled **D**
4430Show errors **DE**
4531============= =========== =================================
4632
33+Configuration defaults (at built time):
34+
4735.. _configured engines:
4836
49−.. jinja:: searx
37+.. jinja:: webapp
5038
5139 .. flat-table:: Engines configured at built time (defaults)
5240 :header-rows: 1
@@ -62,10 +50,10 @@ Show errors **DE**
6250 - SS
6351 - D
6452 - TR
65− - ET
66− - W
67− - D
68− - DE
53+ - O
54+ - W
55+ - D
56+ - DE
6957
7058 {% for name, mod in engines.items() %}
7159
@@ -79,10 +67,9 @@ Show errors **DE**
7967 - {{(mod.safesearch and "y") or ""}}
8068 - {{(mod.disabled and "y") or ""}}
8169 - {{(mod.time_range_support and "y") or ""}}
82− - {{mod.engine_type or ""}}
70+ - {{(mod.offline and "y") or ""}}
8371 - {{mod.weight or 1 }}
8472 - {{(mod.disabled and "y") or ""}}
8573 - {{(mod.display_error_messages and "y") or ""}}
8674
8775 {% endfor %}
88−
docs/admin/engines/recoll.rstdeleted−50
@@ -1,50 +0,0 @@
1−.. _engine recoll:
2−
3−======
4−Recoll
5−======
6−
7−.. sidebar:: info
8−
9− - `Recoll <https://www.lesbonscomptes.com/recoll/>`_
10− - `recoll-webui <https://framagit.org/medoc92/recollwebui.git>`_
11−
12−Recoll_ is a desktop full-text search tool based on Xapian. By itself Recoll_
13−does not offer web or API access, this can be achieved using recoll-webui_
14−
15−
16−
17−Configuration
18−=============
19−
20−You must configure the following settings:
21−
22−``base_url``:
23− Location where recoll-webui can be reached.
24−
25−``mount_prefix``:
26− Location where the file hierarchy is mounted on your *local* filesystem.
27−
28−``dl_prefix``:
29− Location where the file hierarchy as indexed by recoll can be reached.
30−
31−``search_dir``:
32− Part of the indexed file hierarchy to be search, if empty the full domain is
33− searched.
34−
35−
36−Example
37−=======
38−
39−Scenario:
40−
41−#. Recoll indexes a local filesystem mounted in ``/export/documents/reference``,
42−#. the Recoll search inteface can be reached at https://recoll.example.org/ and
43−#. the contents of this filesystem can be reached though https://download.example.org/reference
44−
45−.. code:: yaml
46−
47− base_url: https://recoll.example.org/
48− mount_prefix: /export/documents
49− dl_prefix: https://download.example.org
50− search_dir: ''
docs/admin/filtron.rst+1−1
@@ -173,7 +173,7 @@ Use it along with ``nginx`` with the following example configuration.
173173 location /searx {
174174 proxy_pass http://127.0.0.1:4004/;
175175
176− proxy_set_header Host $host;
176+ proxy_set_header Host $http_host;
177177 proxy_set_header Connection $http_connection;
178178 proxy_set_header X-Real-IP $remote_addr;
179179 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
docs/admin/installation-apache.rst+7−8
@@ -180,6 +180,10 @@ modules and create a `Location`_ configuration for the searx site. In most
180180distributions you have to un-comment the lines in the main configuration file,
181181except in :ref:`The Debian Layout`.
182182
183+To pass the HTTP HOST header
184+With ProxyPreserveHost_ the incoming Host HTTP request header is passed to the
185+proxied host.
186+
183187.. tabs::
184188
185189 .. group-tab:: Ubuntu / debian
@@ -227,11 +231,6 @@ except in :ref:`The Debian Layout`.
227231 LoadModule proxy_module modules/mod_proxy.so
228232 LoadModule proxy_http_module modules/mod_proxy_http.so
229233
230−With ProxyPreserveHost_ the incoming Host HTTP request header is passed to the
231−proxied host.
232−
233−.. _apache searx via filtron plus morty:
234−
235234.. tabs::
236235
237236 .. group-tab:: searx via filtron plus morty
@@ -286,15 +285,15 @@ proxied host.
286285
287286 </Location>
288287
289− For a fully result proxification add :ref:`morty's <searx morty>` **public
290− URL** to your :origin:`searx/settings.yml`:
288+ Note that reverse proxy advised to be used in case of single-user or
289+ low-traffic instances. For a fully result proxification add :ref:`morty's
290+ <searx morty>` **public URL** to your :origin:`searx/settings.yml`:
291291
292292 .. code:: yaml
293293
294294 result_proxy:
295295 # replace example.org with your server's public name
296296 url : https://example.org/morty
297− key : !!binary "insert_your_morty_proxy_key_here"
298297
299298 server:
300299 image_proxy : True
docs/admin/installation-docker.rst+1−1
@@ -51,7 +51,7 @@ It's also possible to build searx from the embedded Dockerfile.
5151
5252 git clone https://github.com/searx/searx.git
5353 cd searx
54− make docker.build
54+ make docker
5555
5656
5757Public instance
docs/admin/installation-nginx.rst+13−15
@@ -9,7 +9,7 @@ Install with nginx
99.. _nginx server configuration:
1010 https://docs.nginx.com/nginx/admin-guide/web-server/web-server/#setting-up-virtual-servers
1111.. _nginx beginners guide:
12− https://nginx.org/en/docs/beginners_guide.html
12+ http://nginx.org/en/docs/beginners_guide.html
1313.. _Getting Started wiki:
1414 https://www.nginx.com/resources/wiki/start/
1515.. _uWSGI support from nginx:
@@ -163,8 +163,6 @@ Started wiki`_ is always a good resource *to keep in the pocket*.
163163 Create configuration at ``/etc/nginx/conf.d/searx`` and place a
164164 symlink to sites-enabled:
165165
166−.. _nginx searx via filtron plus morty:
167−
168166.. tabs::
169167
170168 .. group-tab:: searx via filtron plus morty
@@ -182,7 +180,7 @@ Started wiki`_ is always a good resource *to keep in the pocket*.
182180 location /searx {
183181 proxy_pass http://127.0.0.1:4004/;
184182
185− proxy_set_header Host $host;
183+ proxy_set_header Host $http_host;
186184 proxy_set_header Connection $http_connection;
187185 proxy_set_header X-Real-IP $remote_addr;
188186 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
@@ -190,8 +188,8 @@ Started wiki`_ is always a good resource *to keep in the pocket*.
190188 proxy_set_header X-Script-Name /searx;
191189 }
192190
193− location /searx/static/ {
194− alias /usr/local/searx/searx-src/searx/static/;
191+ location /searx/static {
192+ alias /usr/local/searx/searx-src/searx/static;
195193 }
196194
197195
@@ -205,28 +203,28 @@ Started wiki`_ is always a good resource *to keep in the pocket*.
205203 location /morty {
206204 proxy_pass http://127.0.0.1:3000/;
207205
208− proxy_set_header Host $host;
206+ proxy_set_header Host $http_host;
209207 proxy_set_header Connection $http_connection;
210208 proxy_set_header X-Real-IP $remote_addr;
211209 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
212210 proxy_set_header X-Scheme $scheme;
213211 }
214212
215− For a fully result proxification add :ref:`morty's <searx morty>` **public
216− URL** to your :origin:`searx/settings.yml`:
213+ Note that reverse proxy advised to be used in case of single-user or
214+ low-traffic instances. For a fully result proxification add :ref:`morty's
215+ <searx morty>` **public URL** to your :origin:`searx/settings.yml`:
217216
218217 .. code:: yaml
219218
220219 result_proxy:
221220 # replace example.org with your server's public name
222221 url : https://example.org/morty
223− key : !!binary "insert_your_morty_proxy_key_here"
224222
225223 server:
226224 image_proxy : True
227225
228226
229− .. group-tab:: proxy or uWSGI
227+ .. group-tab:: proxy or uWSGI
230228
231229 Be warned, with this setup, your instance isn't :ref:`protected <searx
232230 filtron>`. Nevertheless it is good enough for intranet usage and it is a
@@ -309,8 +307,8 @@ Started wiki`_ is always a good resource *to keep in the pocket*.
309307 proxy_buffering off;
310308 }
311309
312− location /searx/static/ {
313− alias /usr/local/searx/searx-src/searx/static/;
310+ location /searx/static {
311+ alias /usr/local/searx/searx-src/searx/static;
314312 }
315313
316314 The ``X-Script-Name /searx`` is needed by the searx implementation to
@@ -328,8 +326,8 @@ Started wiki`_ is always a good resource *to keep in the pocket*.
328326 uwsgi_pass unix:/run/uwsgi/app/searx/socket;
329327 }
330328
331− location /searx/static/ {
332− alias /usr/local/searx/searx-src/searx/;
329+ location /searx/static {
330+ alias /usr/local/searx/searx-src/searx;
333331 }
334332
335333 For searx to work correctly the ``base_url`` must be set in the
docs/admin/installation-searx.rst+6−34
@@ -52,7 +52,7 @@ In the same shell create *virtualenv*:
5252 :end-before: END create virtualenv
5353
5454To install searx's dependencies, exit the searx *bash* session you opened above
55−and restart a new. Before install, first check if your *virtualenv* was sourced
55+and restart a new. Before install, first check if your *virualenv* was sourced
5656from the login (*~/.profile*):
5757
5858.. kernel-include:: $DOCS_BUILD/includes/searx.rst
@@ -61,48 +61,20 @@ from the login (*~/.profile*):
6161
6262.. tip::
6363
64− Open a second terminal for the configuration tasks and leave the ``(searx)$``
64+ Open a second terminal for the configuration tasks and left the ``(searx)$``
6565 terminal open for the tasks below.
6666
67−
68−.. _use_default_settings.yml:
69−
7067Configuration
71−=============
72−
73−.. sidebar:: ``use_default_settings: True``
68+==============
7469
75− - :ref:`settings global`
76− - :ref:`settings location`
77− - :ref:`settings use_default_settings`
78− - :origin:`/etc/searx/settings.yml <utils/templates/etc/searx/use_default_settings.yml>`
79−
80−To create a initial ``/etc/searx/settings.yml`` you can start with a copy of the
81−file :origin:`utils/templates/etc/searx/use_default_settings.yml`. This setup
82−:ref:`use default settings <settings use_default_settings>` from
83−:origin:`searx/settings.yml` and is recommended since :pull:`2291` is merged.
84−
85−For a *minimal setup*, configure like shown below – replace ``searx@$(uname
86−-n)`` with a name of your choice, set ``ultrasecretkey`` -- *and/or* edit
87−``/etc/searx/settings.yml`` to your needs.
70+Create a copy of the :origin:`searx/settings.yml` configuration file in system's
71+*/etc* folder. Configure like shown below -- replace ``searx@\$(uname -n)`` with
72+a name of your choice -- *and/or* edit ``/etc/searx/settings.yml`` if necessary.
8873
8974.. kernel-include:: $DOCS_BUILD/includes/searx.rst
9075 :start-after: START searx config
9176 :end-before: END searx config
9277
93−.. tabs::
94−
95− .. group-tab:: Use default settings
96−
97− .. literalinclude:: ../../utils/templates/etc/searx/use_default_settings.yml
98− :language: yaml
99−
100− .. group-tab:: searx/settings.yml
101−
102− .. literalinclude:: ../../searx/settings.yml
103− :language: yaml
104−
105−
10678Check
10779=====
10880
docs/admin/installation.rst−20
@@ -33,24 +33,12 @@ The following will install a setup as shown in :ref:`architecture`. First you
3333need to get a clone. The clone is only needed for the installation procedure
3434and some maintenance tasks (alternatively you can create your own fork).
3535
36−For the installation procedure, use a *sudoer* login to run the scripts. If you
37−install from ``root``, take into account that the scripts are creating a
38−``searx``, a ``filtron`` and a ``morty`` user. In the installation procedure
39−these new created users do need read access to the clone of searx, which is not
40−the case if you clone into a folder below ``/root``.
41−
4236.. code:: bash
4337
4438 $ cd ~/Downloads
4539 $ git clone https://github.com/searx/searx searx
4640 $ cd searx
4741
48−.. sidebar:: further read
49−
50− - :ref:`toolboxing`
51− - :ref:`update searx`
52− - :ref:`inspect searx`
53−
5442**Install** :ref:`searx service <searx.sh>`
5543
5644This installs searx as described in :ref:`installation basic`.
@@ -76,11 +64,3 @@ If all services are running fine, you can add it to your HTTP server:
7664- :ref:`installation apache`
7765- :ref:`installation nginx`
7866
79−.. _git stash: https://git-scm.com/docs/git-stash
80−
81−.. tip::
82−
83− About script's installation options have a look at chapter :ref:`toolboxing
84− setup`. How to brand your instance see chapter :ref:`settings global`. To
85− *stash* your instance's setup, `git stash`_ your clone's :origin:`Makefile`
86− and :origin:`.config.sh` file .
docs/admin/morty.rst+4−11
@@ -16,22 +16,15 @@ By default searx can only act as an image proxy for result images, but it is
1616possible to proxify all the result URLs with an external service, morty_.
1717
1818To use this feature, morty has to be installed and activated in searx's
19−``settings.yml``. Add the following snippet to your ``settings.yml`` and
20−restart searx:
19+``settings.yml``.
20+
21+Add the following snippet to your ``settings.yml`` and restart searx:
2122
2223.. code:: yaml
2324
2425 result_proxy:
2526 url : http://127.0.0.1:3000/
26− key : !!binary "insert_your_morty_proxy_key_here"
27−
28−Note that the example above (``http://127.0.0.1:3000``) is only for single-user
29−instances without a HTTP proxy. If your morty service is public, the url is the
30−address of the reverse proxy (e.g ``https://example.org/morty``).
31−
32−For more information about *result proxy* have a look at *"searx via filtron
33−plus morty"* in the :ref:`nginx <nginx searx via filtron plus morty>` and
34−:ref:`apache <apache searx via filtron plus morty>` sections.
27+ key : your_morty_proxy_key
3528
3629``url``
3730 Is the address of the running morty service.
docs/admin/plugins.rst+1−1
@@ -14,7 +14,7 @@ Configuration defaults (at built time):
1414
1515.. _configured plugins:
1616
17−.. jinja:: searx
17+.. jinja:: webapp
1818
1919 .. flat-table:: Plugins configured at built time (defaults)
2020 :header-rows: 1
docs/admin/settings.rst+64−268
@@ -9,7 +9,6 @@ file.
99
1010.. sidebar:: Further reading ..
1111
12− - :ref:`use_default_settings.yml`
1312 - :ref:`search API`
1413
1514.. contents:: Contents
@@ -17,205 +16,104 @@ file.
1716 :local:
1817 :backlinks: entry
1918
20−.. _settings location:
21−
22−settings.yml location
23−=====================
24−
25−First, searx will try to load settings.yml from these locations:
26−
27−1. the full path specified in the ``SEARX_SETTINGS_PATH`` environment variable.
28−2. ``/etc/searx/settings.yml``
29−
30−If these files don't exist (or are empty or can't be read), searx uses the
31−:origin:`searx/settings.yml` file.
32−
33−
3419.. _settings global:
3520
3621Global Settings
3722===============
3823
39−``general:``
40−------------
41−
4224.. code:: yaml
4325
44− general:
45− debug : False # Debug mode, only for development
46− instance_name : "searx" # displayed name
47− git_url: https://github.com/searx/searx
48− git_branch: master
49− issue_url: https://github.com/searx/searx/issues
50− docs_url: https://searx.github.io/searx
51− public_instances: https://searx.space
52− contact_url: False # mailto:contact@example.com
53− wiki_url: https://github.com/searx/searx/wiki
54− twitter_url: https://twitter.com/Searx_engine
55−
56−``debug`` :
57− Allow a more detailed log if you run searx directly. Display *detailed* error
58− messages in the browser too, so this must be deactivated in production.
59−
60−``contact_url``:
61− Contact ``mailto:`` address or WEB form.
62−
63−``git_url`` and ``git_branch``:
64− Changes this, to point to your searx fork (branch).
26+ server:
27+ port : 8888
28+ secret_key : "ultrasecretkey" # change this!
29+ debug : False # debug mode, only for development
30+ request_timeout : 2.0 # seconds
31+ base_url : False # set custom base_url (or False)
32+ themes_path : "" # custom ui themes path
33+ default_theme : oscar # ui theme
34+ useragent_suffix : "" # suffix of searx_useragent, could contain
35+ # informations like admins email address
36+ image_proxy : False # proxying image results through searx
37+ default_locale : "" # default interface locale
6538
66−``docs_url``
67− If you host your own documentation, change this URL.
39+ # uncomment below section if you want to use a proxy
6840
69−``wiki_url``:
70− Link to your wiki (or ``False``)
41+ #outgoing_proxies :
42+ # http : http://127.0.0.1:8080
43+ # https: http://127.0.0.1:8080
7144
72−``twitter_url``:
73− Link to your tweets (or ``False``)
45+ # uncomment below section only if you have more than one network interface
46+ # which can be the source of outgoing search requests
7447
48+ #source_ips:
49+ # - 1.1.1.1
50+ # - 1.1.1.2
7551
76−``server:``
77−-----------
52+ locales:
53+ en : English
54+ de : Deutsch
55+ he : Hebrew
56+ hu : Magyar
57+ fr : Français
58+ es : Español
59+ it : Italiano
60+ nl : Nederlands
61+ ja : 日本語 (Japanese)
62+ tr : Türkçe
63+ ru : Russian
64+ ro : Romanian
7865
79−.. code:: yaml
8066
81− server:
82− port : 8888
83− bind_address : "127.0.0.1" # address to listen on
84− secret_key : "ultrasecretkey" # change this!
85− base_url : False # set custom base_url (or False)
86− image_proxy : False # proxying image results through searx
87− default_locale : "" # default interface locale
88− default_theme : oscar # ui theme
89− default_http_headers:
90− X-Content-Type-Options : nosniff
91− X-XSS-Protection : 1; mode=block
92− X-Download-Options : noopen
93− X-Robots-Tag : noindex, nofollow
94− Referrer-Policy : no-referrer
95−
96−``port`` & ``bind_address``:
97− Port number and *bind address* of the searx web application if you run it
98− directly using ``python searx/webapp.py``. Doesn't apply to searx running on
99− Apache or Nginx.
67+``port`` :
68+ Port number of the searx web application if you run it directly using ``python
69+ searx/webapp.py``. Doesn't apply to searx running on Apache or Nginx.
10070
10171``secret_key`` :
10272 Used for cryptography purpose.
10373
74+``debug`` :
75+ Allow a more detailed log if you run searx directly. Display *detailed* error
76+ messages in the browser too, so this must be deactivated in production.
77+
78+``request_timeout`` :
79+ Global timeout of the requests made to others engines in seconds. A bigger
80+ timeout will allow to wait for answers from slow engines, but in consequence
81+ will slow searx reactivity (the result page may take the time specified in the
82+ timeout to load)
83+
10484``base_url`` :
10585 The base URL where searx is deployed. Used to create correct inbound links.
10686
107−``image_proxy`` :
108− Allow your instance of searx of being able to proxy images. Uses memory space.
109−
110−``default_locale`` :
111− Searx interface language. If blank, the locale is detected by using the
112− browser language. If it doesn't work, or you are deploying a language
113− specific instance of searx, a locale can be defined using an ISO language
114− code, like ``fr``, ``en``, ``de``.
87+``themes_path`` :
88+ Path to where the themes are located. If you didn't develop anything, leave it
89+ blank.
11590
11691``default_theme`` :
11792 Name of the theme you want to use by default on your searx instance.
11893
119−.. _HTTP headers: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
120−
121−``default_http_headers``:
122− Set additional HTTP headers, see `#755 <https://github.com/searx/searx/issues/715>`__
123−
124−``outgoing:``
125−-------------
126−
127−.. code:: yaml
128−
129− outgoing: # communication with search engines
130− request_timeout : 2.0 # default timeout in seconds, can be override by engine
131− # max_request_timeout: 10.0 # the maximum timeout in seconds
132− useragent_suffix : "" # informations like an email address to the administrator
133− pool_connections : 100 # Maximum number of allowable connections, or None for no limits. The default is 100.
134− pool_maxsize : 10 # Number of allowable keep-alive connections, or None to always allow. The default is 10.
135− enable_http2: True # See https://www.python-httpx.org/http2/
136− # uncomment below section if you want to use a proxy
137− # proxies:
138− # all://:
139− # - http://proxy1:8080
140− # - http://proxy2:8080
141− # uncomment below section only if you have more than one network interface
142− # which can be the source of outgoing search requests
143− # source_ips:
144− # - 1.1.1.1
145− # - 1.1.1.2
146− # - fe80::/126
147−
148−
149−``request_timeout`` :
150− Global timeout of the requests made to others engines in seconds. A bigger
151− timeout will allow to wait for answers from slow engines, but in consequence
152− will slow searx reactivity (the result page may take the time specified in the
153− timeout to load). Can be override by :ref:`settings engine`
154−
15594``useragent_suffix`` :
15695 Suffix to the user-agent searx uses to send requests to others engines. If an
15796 engine wish to block you, a contact info here may be useful to avoid that.
15897
159−``keepalive_expiry``:
160− Number of seconds to keep a connection in the pool. By default 5.0 seconds.
98+``image_proxy`` :
99+ Allow your instance of searx of being able to proxy images. Uses memory space.
161100
162−.. _httpx proxies: https://www.python-httpx.org/advanced/#http-proxying
101+``default_locale`` :
102+ Searx interface language. If blank, the locale is detected by using the
103+ browser language. If it doesn't work, or you are deploying a language
104+ specific instance of searx, a locale can be defined using an ISO language
105+ code, like ``fr``, ``en``, ``de``.
106+
107+.. _requests proxies: http://requests.readthedocs.io/en/latest/user/advanced/#proxies
108+.. _PR SOCKS support: https://github.com/kennethreitz/requests/pull/478
163109
164−``proxies`` :
165− Define one or more proxies you wish to use, see `httpx proxies`_.
166− If there are more than one proxy for one protocol (http, https),
167− requests to the engines are distributed in a round-robin fashion.
110+``outgoing_proxies`` :
111+ Define a proxy you wish to use, see `requests proxies`_. SOCKS proxies are
112+ not supported / see `PR SOCKS support`.
168113
169114``source_ips`` :
170115 If you use multiple network interfaces, define from which IP the requests must
171− be made. Example:
172−
173− * ``0.0.0.0`` any local IPv4 address.
174− * ``::`` any local IPv6 address.
175− * ``192.168.0.1``
176− * ``[ 192.168.0.1, 192.168.0.2 ]`` these two specific IP addresses
177− * ``fe80::60a2:1691:e5a2:ee1f``
178− * ``fe80::60a2:1691:e5a2:ee1f/126`` all IP addresses in this network.
179− * ``[ 192.168.0.1, fe80::/126 ]``
180−
181−``retries`` :
182− Number of retry in case of an HTTP error.
183− On each retry, searx uses an different proxy and source ip.
184−
185−``retry_on_http_error`` :
186− Retry request on some HTTP status code.
187−
188− Example:
189−
190− * ``true`` : on HTTP status code between 400 and 599.
191− * ``403`` : on HTTP status code 403.
192− * ``[403, 429]``: on HTTP status code 403 and 429.
193−
194−``enable_http2`` :
195− Enable by default. Set to ``False`` to disable HTTP/2.
196−
197−``max_redirects`` :
198− 30 by default. Maximum redirect before it is an error.
199−
200−
201−``locales:``
202−------------
203−
204−.. code:: yaml
205−
206− locales:
207− en : English
208− de : Deutsch
209− he : Hebrew
210− hu : Magyar
211− fr : Français
212− es : Español
213− it : Italiano
214− nl : Nederlands
215− ja : 日本語 (Japanese)
216− tr : Türkçe
217− ru : Russian
218− ro : Romanian
116+ be made.
219117
220118``locales`` :
221119 Locales codes and their names. Available translations of searx interface.
@@ -241,22 +139,6 @@ Engine settings
241139 api_key : 'apikey'
242140 disabled : True
243141 language : en_US
244− #enable_http: False
245− #enable_http2: False
246− #retries: 1
247− #retry_on_http_error: True # or 403 or [404, 429]
248− #max_connections: 100
249− #max_keepalive_connections: 10
250− #keepalive_expiry: 5.0
251− #proxies:
252− # http:
253− # - http://proxy1:8080
254− # - http://proxy2:8080
255− # https:
256− # - http://proxy1:8080
257− # - http://proxy2:8080
258− # - socks5://user:password@proxy3:1080
259− # - socks5h://user:password@proxy4:1080
260142
261143``name`` :
262144 Name that will be used across searx to define this engine. In settings, on
@@ -302,93 +184,7 @@ Engine settings
302184``display_error_messages`` : default ``True``
303185 When an engine returns an error, the message is displayed on the user interface.
304186
305−``network``: optional
306− Use the network configuration from another engine.
307− In addition, there are two default networks:
308− * ``ipv4`` set ``local_addresses`` to ``0.0.0.0`` (use only IPv4 local addresses)
309− * ``ipv6`` set ``local_addresses`` to ``::`` (use only IPv6 local addresses)
310−
311187.. note::
312188
313189 A few more options are possible, but they are pretty specific to some
314190 engines, and so won't be described here.
315−
316−
317−.. _settings use_default_settings:
318−
319−use_default_settings
320−====================
321−
322−.. sidebar:: ``use_default_settings: True``
323−
324− - :ref:`settings location`
325− - :ref:`use_default_settings.yml`
326− - :origin:`/etc/searx/settings.yml <utils/templates/etc/searx/use_default_settings.yml>`
327−
328−The user defined ``settings.yml`` is loaded from the :ref:`settings location`
329−and can relied on the default configuration :origin:`searx/settings.yml` using:
330−
331− ``use_default_settings: True``
332−
333−``server:``
334− In the following example, the actual settings are the default settings defined
335− in :origin:`searx/settings.yml` with the exception of the ``secret_key`` and
336− the ``bind_address``:
337−
338− .. code-block:: yaml
339−
340− use_default_settings: True
341− server:
342− secret_key: "uvys6bRhKHUdFF5CqbJonSDSRN8H0sCBziNSrDGNVdpz7IeZhveVart3yvghoKHA"
343− bind_address: "0.0.0.0"
344−
345−``engines:``
346− With ``use_default_settings: True``, each settings can be override in a
347− similar way, the ``engines`` section is merged according to the engine
348− ``name``. In this example, searx will load all the engine and the arch linux
349− wiki engine has a :ref:`token<private engines>`:
350−
351− .. code-block:: yaml
352−
353− use_default_settings: True
354− server:
355− secret_key: "uvys6bRhKHUdFF5CqbJonSDSRN8H0sCBziNSrDGNVdpz7IeZhveVart3yvghoKHA"
356− engines:
357− - name: arch linux wiki
358− tokens: ['$ecretValue']
359−
360−``engines:`` / ``remove:``
361− It is possible to remove some engines from the default settings. The following
362− example is similar to the above one, but searx doesn't load the the google
363− engine:
364−
365− .. code-block:: yaml
366−
367− use_default_settings:
368− engines:
369− remove:
370− - google
371− server:
372− secret_key: "uvys6bRhKHUdFF5CqbJonSDSRN8H0sCBziNSrDGNVdpz7IeZhveVart3yvghoKHA"
373− engines:
374− - name: arch linux wiki
375− tokens: ['$ecretValue']
376−
377−``engines:`` / ``keep_only:``
378− As an alternative, it is possible to specify the engines to keep. In the
379− following example, searx has only two engines:
380−
381− .. code-block:: yaml
382−
383− use_default_settings:
384− engines:
385− keep_only:
386− - google
387− - duckduckgo
388− server:
389− secret_key: "uvys6bRhKHUdFF5CqbJonSDSRN8H0sCBziNSrDGNVdpz7IeZhveVart3yvghoKHA"
390− engines:
391− - name: google
392− tokens: ['$ecretValue']
393− - name: duckduckgo
394− tokens: ['$ecretValue']
docs/admin/update-searx.rst+10−46
@@ -4,56 +4,20 @@
44How to update
55=============
66
7−How to update depends on the :ref:`installation` method. If you have used the
8−:ref:`installation scripts`, use ``update`` command from the scripts.
9−
10−**Update** :ref:`searx service <searx.sh>`
11−
12−.. code:: sh
13−
14− sudo -H ./utils/searx.sh update searx
15−
16−**Update** :ref:`filtron reverse proxy <filtron.sh>`
17−
187.. code:: sh
198
20− sudo -H ./utils/filtron.sh update filtron
21−
22−**Update** :ref:`result proxy <morty.sh>`
23−
24−.. code:: bash
25−
26− sudo -H ./utils/morty.sh update morty
27−
28−.. _inspect searx:
29−
30−======================
31−How to inspect & debug
32−======================
33−
34−.. sidebar:: further read
35−
36− - :ref:`toolboxing`
37− - :ref:`Makefile`
38−
39−How to debug depends on the :ref:`installation` method. If you have used the
40−:ref:`installation scripts`, use ``inspect`` command from the scripts.
41−
42−**Inspect** :ref:`searx service <searx.sh>`
43−
44−.. code:: sh
45−
46− sudo -H ./utils/searx.sh inspect service
47−
48−**Inspect** :ref:`filtron reverse proxy <filtron.sh>`
49−
50−.. code:: sh
9+ sudo -H -u searx -i
10+ (searx)$ git stash
11+ (searx)$ git pull origin master
12+ (searx)$ git stash apply
13+ (searx)$ ./manage.sh update_packages
5114
52− sudo -H ./utils/filtron.sh inspect service
15+Restart uwsgi:
5316
54−**Inspect** :ref:`result proxy <morty.sh>`
17+.. tabs::
5518
56−.. code:: bash
19+ .. group-tab:: Ubuntu / debian
5720
58− sudo -H ./utils/morty.sh inspect service
21+ .. code:: sh
5922
23+ sudo -H systemctl restart uwsgi
docs/blog/index.rst+3−6
@@ -3,15 +3,12 @@ Blog
33====
44
55.. toctree::
6− :titlesonly:
7− :reversed:
6+ :maxdepth: 2
7+ :caption: Contents
88
9+ lxcdev-202006
910 python3
1011 admin
1112 intro-offline
1213 private-engines
13− lxcdev-202006
1414 command-line-engines
15− search-indexer-engines
16− sql-engines
17− search-database-engines
docs/blog/lxcdev-202006.rst+5−5
@@ -31,7 +31,7 @@ might fail in some aspects we should not overlook.
3131
3232 The environment in which we run all our development processes matters!
3333
34−The :ref:`makefile` and the :ref:`make install` encapsulate a lot for us, but they
34+The :ref:`makefile` and the :ref:`make pyenv` encapsulate a lot for us, but they
3535do not have access to all prerequisites. For example, there may have
3636dependencies on packages that are installed on the developer's desktop, but
3737usually are not preinstalled on a server or client system. Another examples
@@ -259,8 +259,8 @@ suite. For this, we have to keep an eye on the :ref:`installation basic`:
259259- virtualenv in: ``/usr/local/searx/searx-pyenv``
260260- searx software in: ``/usr/local/searx/searx-src``
261261
262−The searx software is a clone of the ``git_url`` (see :ref:`settings global`) and
263−the working tree is checked out from the ``git_branch``. With the use of the
262+The searx software is a clone of the ``GIT_URL`` (see :ref:`makefile setup`) and
263+the working tree is checked out from the ``GIT_BRANCH``. With the use of the
264264:ref:`searx.sh` the searx service was installed as :ref:`uWSGI application
265265<searx uwsgi>`. To maintain this service, we can use ``systemctl`` (compare
266266:ref:`service architectures on distributions <uwsgi configuration>`).
@@ -356,7 +356,7 @@ daily usage:
356356 .. code:: sh
357357
358358 $ sudo -H ./utils/lxc.sh cmd searx-archlinux \
359− make docs.html
359+ make docs
360360
361361.. _blog-lxcdev-202006 abstract:
362362
@@ -407,7 +407,7 @@ To get remarks from the suite of the archlinux container we can use:
407407 ...
408408 [searx-archlinux] INFO: (eth0) filtron: http://10.174.184.156:4004/ http://10.174.184.156/searx
409409 [searx-archlinux] INFO: (eth0) morty: http://10.174.184.156:3000/
410− [searx-archlinux] INFO: (eth0) docs.live: http://10.174.184.156:8080/
410+ [searx-archlinux] INFO: (eth0) docs-live: http://10.174.184.156:8080/
411411 [searx-archlinux] INFO: (eth0) IPv6: http://[fd42:573b:e0b3:e97e:216:3eff:fea5:9b65]
412412 ...
413413
docs/blog/private-engines.rst−2
@@ -7,8 +7,6 @@ enabled engines on their instances. It might be because they do not want to
77expose some private information through an offline engine. Or they
88would rather share engines only with their trusted friends or colleagues.
99
10−.. _private engines:
11−
1210Private engines
1311===============
1412
docs/blog/search-database-engines.rstdeleted−95
@@ -1,95 +0,0 @@
1−===============================
2−Query more of your NoSQL stores
3−===============================
4−
5−From now on, searx lets you to query your NoSQL data stores:
6−
7−* `Redis`_
8−* `MongoDB`_
9−
10−The reference configuration of the engines are included ``settings.yml`` just commented out,
11−as you have to set various options and install dependencies before using them.
12−
13−By default, the engines use ``key-value`` template for displaying results.
14−If you are not satisfied with the original result layout,
15−you can use your owm template by placing the template under
16−``searx/templates/{theme_name}/result_templates/{template_name}`` and setting
17−``result_template`` attribute to ``{template_name}``.
18−
19−Futhermore, if you do not want to expose these engines on a public instance, you can
20−still add them and limit the access by setting ``tokens`` as described in the `blog post about
21−private engines`_.
22−
23−Configuring searx to use the stores
24−===================================
25−
26−NoSQL data stores are used for storing arbitrary data without first defining their
27−structure.
28−
29−Redis
30−-----
31−
32−Reqired package: ``redis``
33−
34−Redis is a key value based data store usually stored in memory.
35−
36−Select a database to search in and set its index in the option ``db``. You can
37−either look for exact matches or use partial keywords to find what you are looking for
38−by configuring ``exact_match_only``.
39−
40−In this example you can search for exact matches in your first database:
41−
42−.. code:: yaml
43−
44− - name : mystore
45− engine : redis_server
46− exact_match_only : True
47− host : 127.0.0.1
48− port : 6379
49− password : secret-password
50− db : 0
51− shortcut : rds
52− enable_http : True
53−
54−
55−MongoDB
56−-------
57−
58−Required package: ``pymongo``
59−
60−MongoDB is a document based database program that handles JSON like data.
61−
62−In order to query MongoDB, you have to select a ``database`` and a ``collection``. Furthermore,
63−you have to select a ``key`` that is going to be searched. MongoDB also supports the option ``exact_match_only``, so configure it
64−as you wish.
65−
66−Above is an example configuration for using a MongoDB collection:
67−
68−.. code:: yaml
69−
70− - name : mymongo
71− engine : mongodb
72− shortcut : md
73− host : '127.0.0.1'
74− port : 27017
75− database : personal
76− collection : income
77− key : month
78− enable_http: True
79−
80−
81−Acknowledgement
82−===============
83−
84−This development was sponsored by `Search and Discovery Fund`_ of `NLnet Foundation`_ .
85−
86−.. _Redis: https://redis.io/
87−.. _MongoDB: https://mongodb.com/
88−.. _blog post about private engines: private-engines.html#private-engines
89−.. _Search and Discovery Fund: https://nlnet.nl/discovery
90−.. _NLnet Foundation: https://nlnet.nl/
91−
92−
93−| Happy hacking.
94−| kvch // 2021.07.13 23:16
95−
docs/blog/search-indexer-engines.rstdeleted−114
@@ -1,114 +0,0 @@
1−===============================
2−Query your local search engines
3−===============================
4−
5−From now on, searx lets you to query your locally running search engines. The following
6−ones are supported now:
7−
8−* `Elasticsearch`_
9−* `Meilisearch`_
10−* `Solr`_
11−
12−All of the engines above are added to ``settings.yml`` just commented out, as you have to
13−``base_url`` for all them.
14−
15−Please note that if you are not using HTTPS to access these engines, you have to enable
16−HTTP requests by setting ``enable_http`` to ``True``.
17−
18−Futhermore, if you do not want to expose these engines on a public instance, you can
19−still add them and limit the access by setting ``tokens`` as described in the `blog post about
20−private engines`_.
21−
22−Configuring searx for search engines
23−====================================
24−
25−Each search engine is powerful, capable of full-text search.
26−
27−Elasticsearch
28−-------------
29−
30−Elasticsearch supports numerous ways to query the data it is storing. At the moment
31−the engine supports the most popular search methods: ``match``, ``simple_query_string``, ``term`` and ``terms``.
32−
33−If none of the methods fit your use case, you can select ``custom`` query type and provide the JSON payload
34−searx has to submit to Elasticsearch in ``custom_query_json``.
35−
36−The following is an example configuration for an Elasticsearch instance with authentication
37−configured to read from ``my-index`` index.
38−
39−.. code:: yaml
40−
41− - name : elasticsearch
42− shortcut : es
43− engine : elasticsearch
44− base_url : http://localhost:9200
45− username : elastic
46− password : changeme
47− index : my-index
48− query_type : match
49− enable_http : True
50−
51−
52−Meilisearch
53−-----------
54−
55−This search engine is aimed at individuals and small companies. It is designed for
56−small-scale (less than 10 million documents) data collections. E.g. it is great for storing
57−web pages you have visited and searching in the contents later.
58−
59−The engine supports faceted search, so you can search in a subset of documents of the collection.
60−Futhermore, you can search in Meilisearch instances that require authentication by setting ``auth_token``.
61−
62−Here is a simple example to query a Meilisearch instance:
63−
64−.. code:: yaml
65−
66− - name : meilisearch
67− engine : meilisearch
68− shortcut: mes
69− base_url : http://localhost:7700
70− index : my-index
71− enable_http: True
72−
73−
74−Solr
75−----
76−
77−Solr is a popular search engine based on Lucene, just like Elasticsearch.
78−But instead of searching in indices, you can search in collections.
79−
80−This is an example configuration for searching in the collection ``my-collection`` and get
81−the results in ascending order.
82−
83−.. code:: yaml
84−
85− - name : solr
86− engine : solr
87− shortcut : slr
88− base_url : http://localhost:8983
89− collection : my-collection
90− sort : asc
91− enable_http : True
92−
93−
94−Next steps
95−==========
96−
97−The next step is to add support for various SQL databases.
98−
99−Acknowledgement
100−===============
101−
102−This development was sponsored by `Search and Discovery Fund`_ of `NLnet Foundation`_ .
103−
104−.. _blog post about private engines: private-engines.html#private-engines
105−.. _Elasticsearch: https://www.elastic.co/elasticsearch/
106−.. _Meilisearch: https://www.meilisearch.com/
107−.. _Solr: https://solr.apache.org/
108−.. _Search and Discovery Fund: https://nlnet.nl/discovery
109−.. _NLnet Foundation: https://nlnet.nl/
110−
111−
112−| Happy hacking.
113−| kvch // 2021.04.07 23:16
114−
docs/blog/sql-engines.rstdeleted−117
@@ -1,117 +0,0 @@
1−=================
2−Query SQL servers
3−=================
4−
5−Now you can query SQL servers using searx. The following ones are supported:
6−
7−* `PostgreSQL`_
8−* `MySQL`_
9−* `SQLite`_
10−
11−All of the engines above are added to ``settings.yml`` just commented out, as you have to
12−set the required attributes for the engines, e.g. ``database``. By default, the engines use
13−``key-value`` template for displaying results. If you are not satisfied with the original result layout,
14−you can use your owm template by placing the template under
15−``searx/templates/{theme_name}/result_templates/{template_name}`` and setting
16−``result_template`` attribute to ``{template_name}``.
17−
18−As mentioned in previous blog posts, if you do not wish to expose these engines on a
19−public instance, you can still add them and limit the access by setting ``tokens``
20−as described in the `blog post about private engines`_.
21−
22−Configure the engines
23−=====================
24−
25−The configuration of the new database engines are similar. You must put a valid
26−SELECT SQL query in ``query_str``. At the moment you can only bind at most
27−one parameter in your query. By setting the attribute ``limit`` you can
28−define how many results you want from the SQL server. Basically, it
29−is the same as the LIMIT keyword in SQL.
30−
31−Please, do not include LIMIT or OFFSET in your SQL query as the engines
32−rely on these keywords during paging. If you want to configure the number of returned results
33−use the option ``limit``.
34−
35−PostgreSQL
36−----------
37−
38−PostgreSQL is a powerful and robust open source database.
39−
40−Before configuring the PostgreSQL engine, you must install the dependency ``psychopg2``.
41−
42−You can find an example configuration below:
43−
44−.. code:: yaml
45−
46− - name : postgresql
47− engine : postgresql
48− database : my_database
49− username : searx
50− password : password
51− query_str : 'SELECT * from my_table WHERE my_column = %(query)s'
52− shortcut : psql
53−
54−
55−MySQL
56−-----
57−
58−MySQL is said to be the most popular open source database.
59−
60−Before enabling MySQL engine, you must install the package ``mysql-connector-python``.
61−
62−The authentication plugin is configurable by setting ``auth_plugin`` in the attributes.
63−By default it is set to ``caching_sha2_password``.
64−
65−This is an example configuration for quering a MySQL server:
66−
67−.. code:: yaml
68−
69− - name : mysql
70− engine : mysql_server
71− database : my_database
72− username : searx
73− password : password
74− limit : 5
75− query_str : 'SELECT * from my_table WHERE my_column=%(query)s'
76− shortcut : mysql
77−
78−
79−SQLite
80−------
81−
82−SQLite is a small, fast and reliable SQL database engine. It does not require
83−any extra dependency.
84−
85−You can read from your database ``my_database`` using this example configuration:
86−
87−.. code:: yaml
88−
89− - name : sqlite
90− engine : sqlite
91− shortcut: sq
92− database : my_database
93− query_str : 'SELECT * FROM my_table WHERE my_column=:query'
94−
95−
96−Next steps
97−==========
98−
99−The next step is to add support for more data stores, e.g. Redis and MongoDB.
100−
101−Acknowledgement
102−===============
103−
104−This development was sponsored by `Search and Discovery Fund`_ of `NLnet Foundation`_ .
105−
106−.. _PostgreSQL: https://www.postgresql.org/
107−.. _MySQL: https://www.mysql.com/
108−.. _SQLite: https://www.sqlite.org/index.html
109−.. _blog post about private engines: private-engines.html#private-engines
110−.. _Search and Discovery Fund: https://nlnet.nl/discovery
111−.. _NLnet Foundation: https://nlnet.nl/
112−
113−
114−| Happy hacking.
115−| kvch // 2021.05.23 23:16
116−
117−
docs/build-templates/morty.rst+1−1
@@ -32,7 +32,7 @@
3232 (${SERVICE_USER}) $ mkdir ${SERVICE_HOME}/local
3333 (${SERVICE_USER}) $ wget --progress=bar -O \"${GO_TAR}\" \\
3434 \"${GO_PKG_URL}\"
35− (${SERVICE_USER}) $ tar -C ${SERVICE_HOME}/local -xzf \"${GO_TAR}\"
35+ (${SERVICE_USER}) $ tar -C ${SERVICE_HOME}/local/go -xzf \"${GO_TAR}\"
3636 (${SERVICE_USER}) $ which go
3737 ${SERVICE_HOME}/local/go/bin/go
3838
docs/build-templates/searx.rst+2−19
@@ -116,7 +116,6 @@ ${fedora_build}
116116 pip install -U pip
117117 pip install -U setuptools
118118 pip install -U wheel
119− pip install -U pyyaml
120119
121120 # jump to searx's working tree and install searx into virtualenv
122121 (${SERVICE_USER})$ cd \"$SEARX_SRC\"
@@ -129,28 +128,12 @@ ${fedora_build}
129128
130129.. tabs::
131130
132− .. group-tab:: Use default settings
133−
134− .. code-block:: sh
135−
136− $ sudo -H mkdir -p \"$(dirname ${SEARX_SETTINGS_PATH})\"
137− $ sudo -H cp \"$SEARX_SRC/utils/templates/etc/searx/use_default_settings.yml\" \\
138− \"${SEARX_SETTINGS_PATH}\"
139−
140− .. group-tab:: searx/settings.yml
131+ .. group-tab:: bash
141132
142133 .. code-block:: sh
143134
144135 $ sudo -H mkdir -p \"$(dirname ${SEARX_SETTINGS_PATH})\"
145− $ sudo -H cp \"$SEARX_SRC/searx/settings.yml\" \\
146− \"${SEARX_SETTINGS_PATH}\"
147−
148−.. tabs::
149−
150− .. group-tab:: minimal setup
151−
152− .. code-block:: sh
153−
136+ $ sudo -H cp \"$SEARX_SRC/searx/settings.yml\" \"${SEARX_SETTINGS_PATH}\"
154137 $ sudo -H sed -i -e \"s/ultrasecretkey/\$(openssl rand -hex 16)/g\" \"$SEARX_SETTINGS_PATH\"
155138 $ sudo -H sed -i -e \"s/{instance_name}/searx@\$(uname -n)/g\" \"$SEARX_SETTINGS_PATH\"
156139
docs/conf.py+31−34
@@ -1,16 +1,20 @@
11# -*- coding: utf-8 -*-
2−# SPDX-License-Identifier: AGPL-3.0-or-later
32
43import sys, os
4+from sphinx_build_tools import load_sphinx_config
5+from searx.version import VERSION_STRING
56from pallets_sphinx_themes import ProjectLink
67
7−from searx import brand
8−from searx.version import VERSION_STRING
8+from searx.brand import GIT_URL
9+GIT_BRANCH = os.environ.get("GIT_BRANCH", "master")
10+from searx.brand import SEARX_URL
11+from searx.brand import DOCS_URL
12+
913
1014# Project --------------------------------------------------------------
1115
1216project = u'searx'
13−copyright = u'2015-2021, Adam Tauber, Noémi Ványi'
17+copyright = u'2015-2020, Adam Tauber, Noémi Ványi'
1418author = u'Adam Tauber'
1519release, version = VERSION_STRING, VERSION_STRING
1620highlight_language = 'none'
@@ -23,15 +27,9 @@ numfig = True
2327
2428exclude_patterns = ['build-templates/*.rst']
2529
26−import searx.search
27−import searx.engines
28−import searx.plugins
29−searx.search.initialize()
30+from searx import webapp
3031jinja_contexts = {
31− 'searx': {
32− 'engines': searx.engines.engines,
33− 'plugins': searx.plugins.plugins
34− },
32+ 'webapp': dict(**webapp.__dict__),
3533}
3634
3735# usage:: lorem :patch:`f373169` ipsum
@@ -42,20 +40,20 @@ extlinks['wiki'] = ('https://github.com/searx/searx/wiki/%s', ' ')
4240extlinks['pull'] = ('https://github.com/searx/searx/pull/%s', 'PR ')
4341
4442# links to custom brand
45−extlinks['origin'] = (brand.GIT_URL + '/blob/' + brand.GIT_BRANCH + '/%s', 'git://')
46−extlinks['patch'] = (brand.GIT_URL + '/commit/%s', '#')
47−extlinks['search'] = (brand.SEARX_URL + '/%s', '#')
48−extlinks['docs'] = (brand.DOCS_URL + '/%s', 'docs: ')
43+extlinks['origin'] = (GIT_URL + '/blob/' + GIT_BRANCH + '/%s', 'git://')
44+extlinks['patch'] = (GIT_URL + '/commit/%s', '#')
45+extlinks['search'] = (SEARX_URL + '/%s', '#')
46+extlinks['docs'] = (DOCS_URL + '/%s', 'docs: ')
4947extlinks['pypi'] = ('https://pypi.org/project/%s', 'PyPi: ')
5048extlinks['man'] = ('https://manpages.debian.org/jump?q=%s', '')
5149#extlinks['role'] = (
5250# 'https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-%s', '')
5351extlinks['duref'] = (
54− 'https://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#%s', '')
52+ 'http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#%s', '')
5553extlinks['durole'] = (
56− 'https://docutils.sourceforge.net/docs/ref/rst/roles.html#%s', '')
54+ 'http://docutils.sourceforge.net/docs/ref/rst/roles.html#%s', '')
5755extlinks['dudir'] = (
58− 'https://docutils.sourceforge.net/docs/ref/rst/directives.html#%s', '')
56+ 'http://docutils.sourceforge.net/docs/ref/rst/directives.html#%s', '')
5957extlinks['ctan'] = (
6058 'https://ctan.org/pkg/%s', 'CTAN: ')
6159
@@ -100,21 +98,14 @@ imgmath_font_size = 14
10098# sphinx.ext.imgmath setup END
10199
102100html_theme_options = {"index_sidebar_logo": True}
103−html_context = {"project_links": [] }
104−html_context["project_links"].append(ProjectLink("Blog", "blog/index.html"))
105−if brand.GIT_URL:
106− html_context["project_links"].append(ProjectLink("Source", brand.GIT_URL))
107−if brand.WIKI_URL:
108− html_context["project_links"].append(ProjectLink("Wiki", brand.WIKI_URL))
109−if brand.PUBLIC_INSTANCES:
110− html_context["project_links"].append(ProjectLink("Public instances", brand.PUBLIC_INSTANCES))
111−if brand.TWITTER_URL:
112− html_context["project_links"].append(ProjectLink("Twitter", brand.TWITTER_URL))
113−if brand.ISSUE_URL:
114− html_context["project_links"].append(ProjectLink("Issue Tracker", brand.ISSUE_URL))
115−if brand.CONTACT_URL:
116− html_context["project_links"].append(ProjectLink("Contact", brand.CONTACT_URL))
117−
101+html_context = {
102+ "project_links": [
103+ ProjectLink("Source", GIT_URL),
104+ ProjectLink("Wiki", "https://github.com/searx/searx/wiki"),
105+ ProjectLink("Public instances", "https://searx.space/"),
106+ ProjectLink("Twitter", "https://twitter.com/Searx_engine"),
107+ ]
108+}
118109html_sidebars = {
119110 "**": ["project.html", "relations.html", "searchbox.html"],
120111}
@@ -129,3 +120,9 @@ html_show_sourcelink = False
129120latex_documents = [
130121 (master_doc, "searx-{}.tex".format(VERSION_STRING), html_title, author, "manual")
131122]
123+
124+# ------------------------------------------------------------------------------
125+# Since loadConfig overwrites settings from the global namespace, it has to be
126+# the last statement in the conf.py file
127+# ------------------------------------------------------------------------------
128+load_sphinx_config(globals())
docs/dev/contribution_guide.rst+25−14
@@ -87,7 +87,7 @@ In order to submit a patch, please follow the steps below:
8787
8888- Add yourself to the :origin:`AUTHORS.rst` file.
8989
90−- Choose meaningful commit messages, read `Conventional Commits`_
90+- Choose meaning full commit messages, read `Conventional Commits`_
9191
9292 .. code::
9393
@@ -117,8 +117,8 @@ Translation currently takes place on :ref:`transifex <translation>`.
117117Documentation
118118=============
119119
120−.. _Sphinx: https://www.sphinx-doc.org
121−.. _reST: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
120+.. _Sphinx: http://www.sphinx-doc.org
121+.. _reST: http://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
122122
123123.. sidebar:: The reST sources
124124
@@ -132,11 +132,11 @@ Here is an example which makes a complete rebuild:
132132
133133.. code:: sh
134134
135− $ make docs.clean docs.html
135+ $ make docs-clean docs
136136 ...
137137 The HTML pages are in dist/docs.
138138
139−.. _make docs.live:
139+.. _make docs-live:
140140
141141live build
142142----------
@@ -144,19 +144,19 @@ live build
144144.. _sphinx-autobuild:
145145 https://github.com/executablebooks/sphinx-autobuild/blob/master/README.md
146146
147−.. sidebar:: docs.clean
147+.. sidebar:: docs-clean
148148
149149 It is recommended to assert a complete rebuild before deploying (use
150− ``docs.clean``).
150+ ``docs-clean``).
151151
152152Live build is like WYSIWYG. If you want to edit the documentation, its
153−recommended to use. The Makefile target ``docs.live`` builds the docs, opens
153+recommended to use. The Makefile target ``docs-live`` builds the docs, opens
154154URL in your favorite browser and rebuilds every time a reST file has been
155155changed.
156156
157157.. code:: sh
158158
159− $ make docs.live
159+ $ make docs-live
160160 ...
161161 The HTML pages are in dist/docs.
162162 ... Serving on http://0.0.0.0:8000
@@ -169,7 +169,7 @@ argument. E.g to find and use a free port, use:
169169
170170.. code:: sh
171171
172− $ SPHINXOPTS="--port 0" make docs.live
172+ $ SPHINXOPTS="--port 0" make docs-live
173173 ...
174174 ... Serving on http://0.0.0.0:50593
175175 ...
@@ -180,10 +180,21 @@ argument. E.g to find and use a free port, use:
180180deploy on github.io
181181-------------------
182182
183−To deploy documentation at :docs:`github.io <.>` use Makefile target :ref:`make
184−docs.gh-pages`, which builds the documentation and runs all the needed git add,
185−commit and push:
183+To deploy documentation at :docs:`github.io <.>` use Makefile target
184+:ref:`make gh-pages`, which will builds the documentation, clones searx into a sub
185+folder ``gh-pages``, cleans it, copies the doc build into and runs all the
186+needed git add, commit and push:
186187
187188.. code:: sh
188189
189− $ make docs.clean docs.gh-pages
190+ $ make docs-clean gh-pages
191+ ...
192+ SPHINX docs --> file://<...>/dist/docs
193+ The HTML pages are in dist/docs.
194+ ...
195+ Cloning into 'gh-pages' ...
196+ ...
197+ cd gh-pages; git checkout gh-pages >/dev/null
198+ Switched to a new branch 'gh-pages'
199+ ...
200+ doc available at --> https://searx.github.io/searx
docs/dev/engine_overview.rst+28−63
@@ -18,9 +18,6 @@ engines. Adapters are stored under the folder :origin:`searx/engines`.
1818 :depth: 3
1919 :backlinks: entry
2020
21−
22−.. _general engine configuration:
23−
2421general engine configuration
2522============================
2623
@@ -37,15 +34,15 @@ settings. However, the standard way is the following:
3734engine file
3835-----------
3936
40−======================= =========== ========================================================
37+======================= =========== ===========================================
4138argument type information
42−======================= =========== ========================================================
39+======================= =========== ===========================================
4340categories list pages, in which the engine is working
4441paging boolean support multible pages
42+language_support boolean support language choosing
4543time_range_support boolean support search time range
46−engine_type str ``online`` by default, other possibles values are
47− ``offline``, ``online_dictionnary``, ``online_currency``
48−======================= =========== ========================================================
44+offline boolean engine runs offline
45+======================= =========== ===========================================
4946
5047.. _engine settings:
5148
@@ -58,8 +55,6 @@ argument type information
5855name string name of search-engine
5956engine string name of searx-engine
6057 (filename without ``.py``)
61−enable_http bool enable HTTP
62− (by default only HTTPS is enabled).
6358shortcut string shortcut of search-engine
6459timeout string specific timeout for search-engine
6560display_error_messages boolean display error messages on the web UI
@@ -98,6 +93,7 @@ example code
9893 # engine dependent config
9994 categories = ['general']
10095 paging = True
96+ language_support = True
10197
10298
10399making a request
@@ -112,49 +108,22 @@ passed arguments
112108These arguments can be used to construct the search query. Furthermore,
113109parameters with default value can be redefined for special purposes.
114110
115−If the ``engine_type`` is ``online```:
116−
117−====================== ============== ========================================================================
118−argument type default-value, information
119−====================== ============== ========================================================================
120−url str ``''``
121−method str ``'GET'``
122−headers set ``{}``
123−data set ``{}``
124−cookies set ``{}``
125−verify bool ``True``
126−headers.User-Agent str a random User-Agent
127−category str current category, like ``'general'``
128−safesearch int ``0``, between ``0`` and ``2`` (normal, moderate, strict)
129−time_range Optional[str] ``None``, can be ``day``, ``week``, ``month``, ``year``
130−pageno int current pagenumber
131−language str specific language code like ``'en_US'``, or ``'all'`` if unspecified
132−====================== ============== ========================================================================
133−
134−
135−If the ``engine_type`` is ``online_dictionnary```, in addition to the ``online`` arguments:
136−
137−====================== ============ ========================================================================
138−argument type default-value, information
139−====================== ============ ========================================================================
140−from_lang str specific language code like ``'en_US'``
141−to_lang str specific language code like ``'en_US'``
142−query str the text query without the languages
143−====================== ============ ========================================================================
144−
145−If the ``engine_type`` is ``online_currency```, in addition to the ``online`` arguments:
146−
147111====================== ============ ========================================================================
148112argument type default-value, information
149113====================== ============ ========================================================================
150−amount float the amount to convert
151−from str ISO 4217 code
152−to str ISO 4217 code
153−from_name str currency name
154−to_name str currency name
114+url string ``''``
115+method string ``'GET'``
116+headers set ``{}``
117+data set ``{}``
118+cookies set ``{}``
119+verify boolean ``True``
120+headers.User-Agent string a random User-Agent
121+category string current category, like ``'general'``
122+started datetime current date-time
123+pageno int current pagenumber
124+language string specific language code like ``'en_US'``, or ``'all'`` if unspecified
155125====================== ============ ========================================================================
156126
157−
158127parsed arguments
159128----------------
160129
@@ -162,20 +131,16 @@ The function ``def request(query, params):`` always returns the ``params``
162131variable. Inside searx, the following paramters can be used to specify a search
163132request:
164133
165−=================== =========== ==========================================================================
166−argument type information
167−=================== =========== ==========================================================================
168−url str requested url
169−method str HTTP request method
170−headers set HTTP header information
171−data set HTTP data information
172−cookies set HTTP cookies
173−verify bool Performing SSL-Validity check
174−follow_redirects bool Follow redirects
175−max_redirects int maximum redirects, hard limit
176−soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine
177−raise_for_httperror bool True by default: raise an exception if the HTTP code of response is >= 300
178−=================== =========== ==========================================================================
134+============ =========== =========================================================
135+argument type information
136+============ =========== =========================================================
137+url string requested url
138+method string HTTP request method
139+headers set HTTP header information
140+data set HTTP data information (parsed if ``method != 'GET'``)
141+cookies set HTTP cookies
142+verify boolean Performing SSL-Validity check
143+============ =========== =========================================================
179144
180145
181146example code
@@ -294,7 +259,7 @@ latitude latitude of result (in decimal format)
294259longitude longitude of result (in decimal format)
295260boundingbox boundingbox of result (array of 4. values
296261 ``[lat-min, lat-max, lon-min, lon-max]``)
297−geojson geojson of result (https://geojson.org/)
262+geojson geojson of result (http://geojson.org)
298263osm.type type of osm-object (if OSM-Result)
299264osm.id id of osm-object (if OSM-Result)
300265address.name name of object
docs/dev/makefile.rst+123−124
@@ -1,33 +1,65 @@
11.. _makefile:
22
3−========
4−Makefile
5−========
3+================
4+Makefile Targets
5+================
66
77.. _gnu-make: https://www.gnu.org/software/make/manual/make.html#Introduction
88
99.. sidebar:: build environment
1010
11− Before looking deeper at the targets, first read about :ref:`make
12− install`.
11+ Before looking deeper at the targets, first read about :ref:`makefile setup`
12+ and :ref:`make pyenv`.
1313
1414 To install system requirements follow :ref:`buildhosts`.
1515
16−All relevant build tasks are implemented in :origin:`manage.sh` and for CI or
17−IDE integration a small ``Makefile`` wrapper is available. If you are not
18−familiar with Makefiles, we recommend to read gnu-make_ introduction.
16+With the aim to simplify development cycles, started with :pull:`1756` a
17+``Makefile`` based boilerplate was added. If you are not familiar with
18+Makefiles, we recommend to read gnu-make_ introduction.
1919
2020The usage is simple, just type ``make {target-name}`` to *build* a target.
2121Calling the ``help`` target gives a first overview (``make help``):
2222
2323.. program-output:: bash -c "cd ..; make --no-print-directory help"
2424
25+
2526.. contents:: Contents
2627 :depth: 2
2728 :local:
2829 :backlinks: entry
2930
30−.. _make install:
31+
32+.. _makefile setup:
33+
34+Makefile setup
35+==============
36+
37+.. _git stash: https://git-scm.com/docs/git-stash
38+
39+.. sidebar:: fork & upstream
40+
41+ Commit changes in your (local) branch, fork or whatever, but do not push them
42+ upstream / `git stash`_ is your friend.
43+
44+The main setup is done in the :origin:`Makefile`.
45+
46+.. literalinclude:: ../../Makefile
47+ :start-after: START Makefile setup
48+ :end-before: END Makefile setup
49+
50+:GIT_URL: Changes this, to point to your searx fork.
51+:GIT_BRANCH: Changes this, to point to your searx branch.
52+:SEARX_URL: Changes this, to point to your searx instance.
53+:DOCS_URL: If you host your own (*brand*) documentation, change this URL.
54+
55+If you change any of this build environment variables, you have to run ``make
56+buildenv``::
57+
58+ $ make buildenv
59+ build searx/brand.py
60+ build utils/brand.env
61+
62+.. _make pyenv:
3163
3264Python environment
3365==================
@@ -36,42 +68,31 @@ Python environment
3668
3769 ``source ./local/py3/bin/activate``
3870
39−We do no longer need to build up the virtualenv manually. Jump into your git
40−working tree and release a ``make install`` to get a virtualenv with a
41−*developer install* of searx (:origin:`setup.py`). ::
71+With Makefile we do no longer need to build up the virualenv manually (as
72+described in the :ref:`devquickstart` guide). Jump into your git working tree
73+and release a ``make pyenv``:
74+
75+.. code:: sh
4276
4377 $ cd ~/searx-clone
44− $ make install
45− PYENV [virtualenv] installing ./requirements*.txt into local/py3
46− ...
47− PYENV OK
48− PYENV [install] pip install -e 'searx[test]'
78+ $ make pyenv
79+ PYENV usage: source ./local/py3/bin/activate
4980 ...
50− Successfully installed argparse-1.4.0 searx
51− BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
52− BUILDENV INFO:searx:Initialisation done
53− BUILDENV build utils/brand.env
5481
55−If you release ``make install`` multiple times the installation will only
56−rebuild if the sha256 sum of the *requirement files* fails. With other words:
57−the check fails if you edit the requirements listed in
58−:origin:`requirements-dev.txt` and :origin:`requirements.txt`). ::
82+With target ``pyenv`` a development environment (aka virtualenv) was build up in
83+``./local/py3/``. To make a *developer install* of searx (:origin:`setup.py`)
84+into this environment, use make target ``install``:
85+
86+.. code:: sh
5987
6088 $ make install
61− PYENV OK
62− PYENV [virtualenv] requirements.sha256 failed
63− [virtualenv] - 6cea6eb6def9e14a18bf32f8a3e... ./requirements-dev.txt
64− [virtualenv] - 471efef6c73558e391c3adb35f4... ./requirements.txt
65− ...
66− PYENV [virtualenv] installing ./requirements*.txt into local/py3
67− ...
68− PYENV OK
69− PYENV [install] pip install -e 'searx[test]'
70− ...
71− Successfully installed argparse-1.4.0 searx
72− BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
73− BUILDENV INFO:searx:Initialisation done
74− BUILDENV build utils/brand.env
89+ PYENV usage: source ./local/py3/bin/activate
90+ PYENV using virtualenv from ./local/py3
91+ PYENV install .
92+
93+You have never to think about intermediate targets like ``pyenv`` or
94+``install``, the ``Makefile`` chains them as requisites. Just run your main
95+target.
7596
7697.. sidebar:: drop environment
7798
@@ -79,7 +100,10 @@ the check fails if you edit the requirements listed in
79100 <make clean>` first.
80101
81102If you think, something goes wrong with your ./local environment or you change
82−the :origin:`setup.py` file, you have to call :ref:`make clean`.
103+the :origin:`setup.py` file (or the requirements listed in
104+:origin:`requirements-dev.txt` and :origin:`requirements.txt`), you have to call
105+:ref:`make clean`.
106+
83107
84108.. _make run:
85109
@@ -89,113 +113,88 @@ the :origin:`setup.py` file, you have to call :ref:`make clean`.
89113To get up a running a developer instance simply call ``make run``. This enables
90114*debug* option in :origin:`searx/settings.yml`, starts a ``./searx/webapp.py``
91115instance, disables *debug* option again and opens the URL in your favorite WEB
92−browser (:man:`xdg-open`)::
116+browser (:man:`xdg-open`):
93117
94− $ make run
95− PYENV OK
96− SEARX_DEBUG=1 ./manage.sh pyenv.cmd python ./searx/webapp.py
97− ...
98− INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
118+.. code:: sh
119+
120+ $ make run
121+ PYENV usage: source ./local/py3/bin/activate
122+ PYENV install .
123+ ./local/py3/bin/python ./searx/webapp.py
124+ ...
125+ INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
126+ ...
99127
100128.. _make clean:
101129
102130``make clean``
103131==============
104132
105−Drop all intermediate files, all builds, but keep sources untouched. Before
106−calling ``make clean`` stop all processes using :ref:`make install`. ::
133+Drop all intermediate files, all builds, but keep sources untouched. Includes
134+target ``pyclean`` which drops ./local environment. Before calling ``make
135+clean`` stop all processes using :ref:`make pyenv`.
136+
137+.. code:: sh
107138
108139 $ make clean
109− CLEAN pyenv
110− PYENV [virtualenv] drop ./local/py3
111− CLEAN docs -- ./build/docs ./dist/docs
112− CLEAN locally installed npm dependencies
113− CLEAN test stuff
114− CLEAN common files
140+ CLEAN pyclean
141+ CLEAN clean
115142
116143.. _make docs:
117144
118−``make docs docs.autobuild docs.clean``
119−=======================================
145+``make docs docs-live docs-clean``
146+==================================
120147
121−We describe the usage of the ``doc.*`` targets in the :ref:`How to contribute /
148+We describe the usage of the ``doc*`` targets in the :ref:`How to contribute /
122149Documentation <contrib docs>` section. If you want to edit the documentation
123−read our :ref:`make docs.live` section. If you are working in your own brand,
124−adjust your :ref:`settings global`.
150+read our :ref:`make docs-live` section. If you are working in your own brand,
151+adjust your :ref:`Makefile setup <makefile setup>`.
152+
125153
126−.. _make docs.gh-pages:
154+.. _make gh-pages:
127155
128−``make docs.gh-pages``
129−======================
156+``make gh-pages``
157+=================
130158
131−To deploy on github.io first adjust your :ref:`settings global`. For any
132−further read :ref:`deploy on github.io`.
159+To deploy on github.io first adjust your :ref:`Makefile setup <makefile
160+setup>`. For any further read :ref:`deploy on github.io`.
133161
134162.. _make test:
135163
136164``make test``
137165=============
138166
139−Runs a series of tests: :ref:`make test.pylint`, ``test.pep8``, ``test.unit``
140−and ``test.robot``. You can run tests selective, e.g.::
141−
142− $ make test.pep8 test.unit test.sh
143− TEST test.pep8 OK
144− ...
145− TEST test.unit OK
146− ...
147− TEST test.sh OK
167+Runs a series of tests: ``test.pep8``, ``test.unit``, ``test.robot`` and does
168+additional :ref:`pylint checks <make pylint>`. You can run tests selective,
169+e.g.:
148170
149−.. _make test.sh:
171+.. code:: sh
150172
151−``make test.sh``
152−================
153−
154−:ref:`sh lint` / if you have changed some bash scripting run this test before
155−commit.
173+ $ make test.pep8 test.unit test.sh
174+ . ./local/py3/bin/activate; ./manage.sh pep8_check
175+ [!] Running pep8 check
176+ . ./local/py3/bin/activate; ./manage.sh unit_tests
177+ [!] Running unit tests
156178
157−.. _make test.pylint:
179+.. _make pylint:
158180
159−``make test.pylint``
160−====================
181+``make pylint``
182+===============
161183
162184.. _Pylint: https://www.pylint.org/
163185
164−Pylint_ is known as one of the best source-code, bug and quality checker for the
165−Python programming language. The pylint profile we use at searx project is
166−found in project's root folder :origin:`.pylintrc`.
186+Before commiting its recommend to do some (more) linting. Pylint_ is known as
187+one of the best source-code, bug and quality checker for the Python programming
188+language. Pylint_ is not yet a quality gate within our searx project (like
189+:ref:`test.pep8 <make test>` it is), but Pylint_ can help to improve code
190+quality anyway. The pylint profile we use at searx project is found in
191+project's root folder :origin:`.pylintrc`.
167192
168−.. _make search.checker:
169−
170−``search.checker.{engine name}``
171−================================
172−
173−To check all engines::
174−
175− make search.checker
176−
177−To check a engine with whitespace in the name like *google news* replace space
178−by underline::
179−
180− make search.checker.google_news
181−
182−To see HTTP requests and more use SEARX_DEBUG::
183−
184− make SEARX_DEBUG=1 search.checker.google_news
185−
186−.. _3xx: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_redirection
187−
188−To filter out HTTP redirects (3xx_)::
189−
190− make SEARX_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]"
191− ...
192− Engine google news Checking
193− https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
194− https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
195− --
196− https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
197− https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
198− --
193+Code quality is a ongoing process. Don't try to fix all messages from Pylint,
194+run Pylint and check if your changed lines are bringing up new messages. If so,
195+fix it. By this, code quality gets incremental better and if there comes the
196+day, the linting is balanced out, we might decide to add Pylint as a quality
197+gate.
199198
200199
201200``make pybuild``
@@ -204,7 +203,9 @@ To filter out HTTP redirects (3xx_)::
204203.. _PyPi: https://pypi.org/
205204.. _twine: https://twine.readthedocs.io/en/latest/
206205
207−Build Python packages in ``./dist/py``::
206+Build Python packages in ``./dist/py``.
207+
208+.. code:: sh
208209
209210 $ make pybuild
210211 ...
@@ -212,11 +213,9 @@ Build Python packages in ``./dist/py``::
212213 running sdist
213214 running egg_info
214215 ...
215− running bdist_wheel
216−
217− $ ls ./dist
218− searx-0.18.0-py3-none-any.whl searx-0.18.0.tar.gz
216+ $ ls ./dist/py/
217+ searx-0.15.0-py3-none-any.whl searx-0.15.0.tar.gz
219218
220−To upload packages to PyPi_, there is also a ``pypi.upload`` target (to test use
221−``pypi.upload.test``). Since you are not the owner of :pypi:`searx` you will
222−never need to upload.
219+To upload packages to PyPi_, there is also a ``upload-pypi`` target. It needs
220+twine_ to be installed. Since you are not the owner of :pypi:`searx` you will
221+never need the latter.
docs/dev/quickstart.rst+102−25
@@ -4,52 +4,129 @@
44Development Quickstart
55======================
66
7−.. _npm: https://www.npmjs.com/
7+.. sidebar:: :ref:`makefile`
88
9−Searx loves developers, just clone and start hacking. All the rest is done for
10−you simply by using :ref:`make <makefile>`.
9+ For additional developer purpose there are :ref:`makefile`.
10+
11+This quickstart guide gets your environment set up with searx. Furthermore, it
12+gives a short introduction to the ``manage.sh`` script.
13+
14+How to setup your development environment
15+=========================================
16+
17+.. sidebar:: :ref:`make pyenv <make pyenv>`
18+
19+ Alternatively use the :ref:`make pyenv`.
20+
21+First, clone the source code of searx to the desired folder. In this case the
22+source is cloned to ``~/myprojects/searx``. Then create and activate the
23+searx-ve virtualenv and install the required packages using ``manage.sh``.
1124
1225.. code:: sh
1326
27+ cd ~/myprojects
1428 git clone https://github.com/searx/searx.git
29+ cd searx
30+ python3 -m venv searx-ve
31+ . ./searx-ve/bin/activate
32+ ./manage.sh update_dev_packages
1533
16−Here is how a minimal workflow looks like:
1734
18−1. *start* hacking
19−2. *run* your code: :ref:`make run`
20−3. *test* your code: :ref:`make test`
35+How to run tests
36+================
2137
22−If you think at some point something fails, go back to *start*. Otherwise,
23−choose a meaningful commit message and we are happy to receive your pull
24−request. To not end in *wild west* we have some directives, please pay attention
25−to our ":ref:`how to contribute`" guideline.
38+.. sidebar:: :ref:`make test.unit <make test>`
2639
27−If you implement themes, you will need to compile styles and JavaScript before
28−*run*.
40+ Alternatively use the ``test.pep8``, ``test.unit``, ``test.robot`` targets.
41+
42+Tests can be run using the ``manage.sh`` script. Following tests and checks are
43+available:
44+
45+- Unit tests
46+- Selenium tests
47+- PEP8 validation
48+- Unit test coverage check
49+
50+For example unit tests are run with the command below:
2951
3052.. code:: sh
3153
32− make themes
54+ ./manage.sh unit_tests
55+
56+For further test options, please consult the help of the ``manage.sh`` script or
57+read :ref:`make test`.
58+
59+
60+How to compile styles and javascript
61+====================================
62+
63+.. _less: http://lesscss.org/
64+.. _NodeJS: https://nodejs.org
65+
66+How to build styles
67+-------------------
68+
69+Less_ is required to build the styles of searx. Less_ can be installed using
70+either NodeJS_ or Apt.
71+
72+.. code:: sh
3373
34−Don't forget to install npm_ first.
74+ sudo -H apt-get install nodejs
75+ sudo -H npm install -g less
3576
36−.. tabs::
77+OR
3778
38− .. group-tab:: Ubuntu / debian
79+.. code:: sh
80+
81+ sudo -H apt-get install node-less
82+
83+After satisfying the requirements styles can be build using ``manage.sh``
84+
85+.. code:: sh
86+
87+ ./manage.sh styles
88+
89+
90+How to build the source of the themes
91+=====================================
92+
93+.. _grunt: https://gruntjs.com/
3994
40− .. code:: sh
95+Grunt_ must be installed in order to build the javascript sources. It depends on
96+NodeJS, so first Node has to be installed.
97+
98+.. code:: sh
99+
100+ sudo -H apt-get install nodejs
101+ make node.env
102+
103+After installing grunt, the files can be built using the following command:
104+
105+.. code:: sh
106+
107+ make themes
41108
42− sudo -H apt-get install npm
43109
44− .. group-tab:: Arch Linux
110+Tips for debugging/development
111+==============================
45112
46− .. code-block:: sh
113+.. sidebar:: :ref:`make run`
47114
48− sudo -H pacman -S npm
115+ Makefile target ``run`` already enables debug option for your developer
116+ session / see :ref:`make run`.
49117
50− .. group-tab:: Fedora / RHEL
118+Turn on debug logging
119+ Whether you are working on a new engine or trying to eliminate a bug, it is
120+ always a good idea to turn on debug logging. When debug logging is enabled a
121+ stack trace appears, instead of the cryptic ``Internal Server Error``
122+ message. It can be turned on by setting ``debug: False`` to ``debug: True`` in
123+ :origin:`settings.yml <searx/settings.yml>`.
51124
52− .. code-block:: sh
125+.. sidebar:: :ref:`make test`
53126
54− sudo -H dnf install npm
127+ Alternatively use the :ref:`make test` targets.
55128
129+Run ``./manage.sh tests`` before creating a PR.
130+ Failing build on Travis is common because of PEP8 checks. So a new commit
131+ must be created containing these format fixes. This phase can be skipped if
132+ ``./manage.sh tests`` is run locally before creating a PR.
docs/dev/reST.rst+24−32
@@ -15,8 +15,8 @@ generated and deployed at :docs:`github.io <.>`. For build prerequisites read
1515:ref:`docs build`.
1616
1717The source files of Searx's documentation are located at :origin:`docs`. Sphinx
18−assumes source files to be encoded in UTF-8 by defaul. Run :ref:`make docs.live
19−<make docs.live>` to build HTML while editing.
18+assumes source files to be encoded in UTF-8 by defaul. Run :ref:`make docs-live
19+<make docs-live>` to build HTML while editing.
2020
2121.. sidebar:: Further reading
2222
@@ -319,9 +319,6 @@ To list all anchors of the inventory (e.g. ``python``) use:
319319.. code:: sh
320320
321321 $ python -m sphinx.ext.intersphinx https://docs.python.org/3/objects.inv
322− ...
323− $ python -m sphinx.ext.intersphinx https://searx.github.io/searx/objects.inv
324− ...
325322
326323Literal blocks
327324==============
@@ -1276,33 +1273,28 @@ Templating
12761273
12771274.. sidebar:: Build environment
12781275
1279− All *generic-doc* tasks are running in the :ref:`make install`.
1276+ All *generic-doc* tasks are running in the :ref:`build environment <make
1277+ pyenv>`.
12801278
12811279Templating is suitable for documentation which is created generic at the build
1282−time. The sphinx-jinja_ extension evaluates jinja_ templates in the :ref:`make
1283−install` (with searx modules installed). We use this e.g. to build chapter:
1284−:ref:`engines generic`. Below the jinja directive from the
1280+time. The sphinx-jinja_ extension evaluates jinja_ templates in the :ref:`build
1281+environment <make pyenv>` (with searx modules installed). We use this e.g. to
1282+build chapter: :ref:`engines generic`. Below the jinja directive from the
12851283:origin:`docs/admin/engines.rst` is shown:
12861284
12871285.. literalinclude:: ../admin/engines.rst
12881286 :language: reST
12891287 :start-after: .. _configured engines:
12901288
1291−The context for the template is selected in the line ``.. jinja:: searx``. In
1292−sphinx's build configuration (:origin:`docs/conf.py`) the ``searx`` context
1293−contains the ``engines`` and ``plugins``.
1289+The context for the template is selected in the line ``.. jinja:: webapp``. In
1290+sphinx's build configuration (:origin:`docs/conf.py`) the ``webapp`` context
1291+points to the name space of the python module: ``webapp``.
12941292
12951293.. code:: py
12961294
1297− import searx.search
1298− import searx.engines
1299− import searx.plugins
1300− searx.search.initialize()
1295+ from searx import webapp
13011296 jinja_contexts = {
1302− 'searx': {
1303− 'engines': searx.engines.engines,
1304− 'plugins': searx.plugins.plugins
1305− },
1297+ 'webapp': dict(**webapp.__dict__)
13061298 }
13071299
13081300
@@ -1399,27 +1391,27 @@ The next example shows the difference of ``\tfrac`` (*textstyle*) and ``\dfrac``
13991391
14001392.. _readability: https://docs.python-guide.org/writing/style/
14011393.. _Sphinx-Primer:
1402− https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
1394+ http://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
14031395.. _reST: https://docutils.sourceforge.io/rst.html
14041396.. _Sphinx Roles:
14051397 https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html
1406−.. _Sphinx: https://www.sphinx-doc.org
1407−.. _`sphinx-doc FAQ`: https://www.sphinx-doc.org/en/stable/faq.html
1398+.. _Sphinx: http://www.sphinx-doc.org
1399+.. _`sphinx-doc FAQ`: http://www.sphinx-doc.org/en/stable/faq.html
14081400.. _Sphinx markup constructs:
1409− https://www.sphinx-doc.org/en/stable/markup/index.html
1401+ http://www.sphinx-doc.org/en/stable/markup/index.html
14101402.. _`sphinx cross references`:
1411− https://www.sphinx-doc.org/en/stable/markup/inline.html#cross-referencing-arbitrary-locations
1403+ http://www.sphinx-doc.org/en/stable/markup/inline.html#cross-referencing-arbitrary-locations
14121404.. _sphinx.ext.extlinks:
14131405 https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html
1414−.. _intersphinx: https://www.sphinx-doc.org/en/stable/ext/intersphinx.html
1415−.. _sphinx config: https://www.sphinx-doc.org/en/stable/config.html
1416−.. _Sphinx's autodoc: https://www.sphinx-doc.org/en/stable/ext/autodoc.html
1406+.. _intersphinx: http://www.sphinx-doc.org/en/stable/ext/intersphinx.html
1407+.. _sphinx config: http://www.sphinx-doc.org/en/stable/config.html
1408+.. _Sphinx's autodoc: http://www.sphinx-doc.org/en/stable/ext/autodoc.html
14171409.. _Sphinx's Python domain:
1418− https://www.sphinx-doc.org/en/stable/domains.html#the-python-domain
1410+ http://www.sphinx-doc.org/en/stable/domains.html#the-python-domain
14191411.. _Sphinx's C domain:
1420− https://www.sphinx-doc.org/en/stable/domains.html#cross-referencing-c-constructs
1412+ http://www.sphinx-doc.org/en/stable/domains.html#cross-referencing-c-constructs
14211413.. _doctree:
1422− https://www.sphinx-doc.org/en/master/extdev/tutorial.html?highlight=doctree#build-phases
1414+ http://www.sphinx-doc.org/en/master/extdev/tutorial.html?highlight=doctree#build-phases
14231415.. _docutils: http://docutils.sourceforge.net/docs/index.html
14241416.. _docutils FAQ: http://docutils.sourceforge.net/FAQ.html
14251417.. _linuxdoc: https://return42.github.io/linuxdoc
@@ -1432,5 +1424,5 @@ The next example shows the difference of ``\tfrac`` (*textstyle*) and ``\dfrac``
14321424.. _ImageMagick: https://www.imagemagick.org
14331425
14341426.. _`Emacs Table Mode`: https://www.emacswiki.org/emacs/TableMode
1435−.. _`Online Tables Generator`: https://www.tablesgenerator.com/text_tables
1427+.. _`Online Tables Generator`: http://www.tablesgenerator.com/text_tables
14361428.. _`OASIS XML Exchange Table Model`: https://www.oasis-open.org/specs/tm9901.html
docs/dev/search_api.rst+2−2
@@ -6,7 +6,7 @@ Search API
66
77The search supports both ``GET`` and ``POST``.
88
9−Furthermore, two endpoints ``/`` and ``/search`` are available for querying.
9+Furthermore, two enpoints ``/`` and ``/search`` are available for querying.
1010
1111
1212``GET /``
@@ -76,7 +76,7 @@ Parameters
7676 supports safe search in the preferences page of an instance.
7777
7878``theme`` : default ``oscar``
79− [ ``oscar``, ``simple`` ]
79+ [ ``oscar``, ``simple``, ``legacy``, ``pix-art``, ``courgette`` ]
8080
8181 Theme of instance.
8282
docs/index.rst−1
@@ -30,7 +30,6 @@ anyone, you can set up your own, see :ref:`installation`.
3030 user/index
3131 admin/index
3232 dev/index
33− searx_extra/index
3433 utils/index
3534 blog/index
3635
docs/searx_extra/index.rstdeleted−14
@@ -1,14 +0,0 @@
1−.. _searx_extra:
2−
3−======================================================
4−Tooling box ``searx_extra`` for developers and users
5−======================================================
6−
7−In the folder :origin:`searx_extra/` we maintain some tools useful for
8−developers and users.
9−
10−.. toctree::
11− :maxdepth: 2
12− :caption: Contents
13−
14− standalone_searx.py
docs/searx_extra/standalone_searx.py.rstdeleted−9
@@ -1,9 +0,0 @@
1−
2−.. _standalone_searx.py:
3−
4−===================================
5−``searx_extra/standalone_searx.py``
6−===================================
7−
8−.. automodule:: searx_extra.standalone_searx
9− :members:
docs/user/conf.pynew file+19
@@ -0,0 +1,19 @@
1+# -*- coding: utf-8; mode: python -*-
2+"""Configuration for the Searx user handbook
3+"""
4+project = 'Searx User-HB'
5+version = release = VERSION_STRING
6+
7+# Grouping the document tree into LaTeX files. List of tuples
8+# (source start file, target name, title,
9+# author, documentclass [howto, manual, or own class]).
10+latex_documents = [
11+ ('index' # startdocname
12+ , 'searx-user-hb.tex' # targetname
13+ , '' # take title from .rst
14+ , author # author
15+ , 'howto' # documentclass
16+ , False # toctree_only
17+ ),
18+]
19+
docs/user/own-instance.rst+3−3
@@ -56,9 +56,9 @@ results.
5656I see. What about private instances?
5757------------------------------------
5858
59−If users run their :ref:`own instances <installation>`, everything is in their
60−control: the source code, logging settings and private data. Unknown instance
61−administrators do not have to be trusted.
59+If users run their own instances, everything is in their control: the source
60+code, logging settings and private data. Unknown instance administrators do not
61+have to be trusted.
6262
6363Furthermore, as the default settings of their instance is editable, there is no
6464need to use cookies to tailor searx to their needs. So preferences will not be
docs/user/search_syntax.rst−24
@@ -40,27 +40,3 @@ Image search:
4040Custom language in wikipedia:
4141
4242- :search:`:hu !wp hackerspace <?q=%3Ahu%20%21wp%20hackerspace>`
43−
44−Multilingual Search
45−===================
46−
47−Searx does not support true multilingual search.
48−You have to use the language prefix in your search query when searching in a different language.
49−
50−But there is a workaround:
51−By adding a new search engine with a different language, Searx will search in your default and other language.
52−
53−Example configuration in settings.yml for a German and English speaker:
54− .. code-block:: yaml
55−
56− search:
57− language : "de"
58− ...
59−
60− engines:
61− - name : google english
62− engine : google
63− language : english
64− ...
65−
66−When searching, the default google engine will return German results and "google english" will return English results.
docs/utils/index.rst+9−7
@@ -1,11 +1,12 @@
11.. _searx_utils:
22.. _toolboxing:
33
4−===================
5−Admin's tooling box
6−===================
4+=======================
5+Tooling box ``utils/*``
6+=======================
77
8−In the folder :origin:`utils/` we maintain some tools useful for administrators.
8+In the folder :origin:`utils/` we maintain some tools useful for admins and
9+developers.
910
1011.. toctree::
1112 :maxdepth: 2
@@ -15,6 +16,7 @@ In the folder :origin:`utils/` we maintain some tools useful for administrators.
1516 filtron.sh
1617 morty.sh
1718 lxc.sh
19+ standalone_searx.py
1820
1921.. _toolboxing common:
2022
@@ -39,14 +41,14 @@ Scripts to maintain services often dispose of common commands and environments.
3941 <lxc-searx.env>` use ::
4042
4143 sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/filtron.sh apache install
42−
44+
4345.. _toolboxing setup:
4446
4547Tooling box setup
4648=================
4749
48−The main setup is done in the :origin:`.config.sh` (read also :ref:`settings
49−global`).
50+The main setup is done in the :origin:`.config.sh` (read also :ref:`makefile
51+setup`).
5052
5153.. literalinclude:: ../../.config.sh
5254 :language: bash
docs/utils/lxc.sh.rst+3−3
@@ -119,15 +119,15 @@ of coffee).::
119119
120120To build (live) documentation inside a archlinux_ container::
121121
122− sudo -H ./utils/lxc.sh cmd searx-archlinux make docs.clean docs.live
122+ sudo -H ./utils/lxc.sh cmd searx-archlinux make docs-clean docs-live
123123 ...
124124 [I 200331 15:00:42 server:296] Serving on http://0.0.0.0:8080
125125
126126To get IP of the container and the port number *live docs* is listening::
127127
128− $ sudo ./utils/lxc.sh show suite | grep docs.live
128+ $ sudo ./utils/lxc.sh show suite | grep docs-live
129129 ...
130− [searx-archlinux] INFO: (eth0) docs.live: http://n.n.n.12:8080/
130+ [searx-archlinux] INFO: (eth0) docs-live: http://n.n.n.12:8080/
131131
132132
133133.. _lxc.sh help:
docs/utils/standalone_searx.py.rstnew file+11
@@ -0,0 +1,11 @@
1+
2+.. _standalone_searx.py:
3+
4+=============================
5+``utils/standalone_searx.py``
6+=============================
7+
8+.. automodule:: standalone_searx
9+ :members:
10+
11+
managedeleted−504
@@ -1,504 +0,0 @@
1−#!/usr/bin/env bash
2−# -*- coding: utf-8; mode: sh indent-tabs-mode: nil -*-
3−# SPDX-License-Identifier: AGPL-3.0-or-later
4−# shellcheck disable=SC2031
5−
6−# shellcheck source=utils/lib.sh
7−source "$(dirname "${BASH_SOURCE[0]}")/utils/lib.sh"
8−# shellcheck source=utils/brand.env
9−source "${REPO_ROOT}/utils/brand.env"
10−source_dot_config
11−
12−# config
13−
14−PY_SETUP_EXTRAS='[test]'
15−NPM_PACKAGES="less@2.7 less-plugin-clean-css grunt-cli"
16−GECKODRIVER_VERSION="v0.30.0"
17−# SPHINXOPTS=
18−
19−# These py files are linted by test.pylint(), all other files are linted by
20−# test.pep8()
21−PYLINT_FILES=(
22− searx/preferences.py
23− searx/testing.py
24− searx/engines/gigablast.py
25− searx/engines/deviantart.py
26− searx/engines/digg.py
27− searx/engines/google.py
28− searx/engines/google_news.py
29− searx/engines/google_videos.py
30− searx/engines/google_images.py
31− searx/engines/mediathekviewweb.py
32− searx/engines/meilisearch.py
33− searx/engines/solidtorrents.py
34− searx/engines/solr.py
35− searx/engines/sqlite.py
36− searx/engines/springer.py
37− searx/engines/google_scholar.py
38− searx/engines/yahoo_news.py
39− searx/engines/apkmirror.py
40− searx/engines/core.py
41− searx_extra/update/update_external_bangs.py
42−)
43−
44−PYLINT_SEARX_DISABLE_OPTION="\
45−I,C,R,\
46−W0105,W0212,W0511,W0603,W0613,W0621,W0702,W0703,W1401,\
47−E1136"
48−PYLINT_ADDITIONAL_BUILTINS_FOR_ENGINES="supported_languages,language_aliases"
49−PYLINT_OPTIONS="-m pylint -j 0 --rcfile .pylintrc"
50−
51−help() {
52− cat <<EOF
53−buildenv
54− rebuild ./utils/brand.env
55−babel.compile
56− pybabel compile ./searx/translations
57−data.*
58− all : update searx/languages.py and ./data/*
59− languages : update searx/data/engines_languages.json & searx/languages.py
60− useragents: update searx/data/useragents.json with the most recent versions of Firefox.
61−docs.*
62− html : build HTML documentation
63− live : autobuild HTML documentation while editing
64− gh-pages : deploy on gh-pages branch
65− prebuild : build reST include files (./${DOCS_BUILD}/includes)
66− clean : clean documentation build
67−docker
68− build : build docker image
69− push : build and push docker image
70−gecko.driver
71− download & install geckodriver if not already installed (required for
72− robot_tests)
73−node.*
74− env : download & install npm dependencies locally
75− clean : drop npm installations
76−py.*
77− build : Build python packages at ./${PYDIST}
78− clean : delete virtualenv and intermediate py files
79−pyenv.* :
80− install : developer install of searx into virtualenv
81− uninstall : uninstall developer installation
82− cmd ... : run command ... in virtualenv
83− OK : test if virtualenv is OK
84−pypi.upload:
85− Upload python packages to PyPi (to test use pypi.upload.test)
86−test.* :
87− pylint : lint PYLINT_FILES, searx/engines, searx & tests
88− pep8 : pycodestyle (pep8) for all files except PYLINT_FILES
89− unit : run unit tests
90− coverage : run unit tests with coverage
91− robot : run robot test
92− clean : clean intermediate test stuff
93−themes.* :
94− all : build all themes
95− oscar : build oscar theme
96− simple : build simple theme
97−EOF
98−}
99−
100−
101−if [ "$VERBOSE" = "1" ]; then
102− SPHINX_VERBOSE="-v"
103− PYLINT_VERBOSE="-v"
104−fi
105−
106−# needed by sphinx-docs
107−export DOCS_BUILD
108−
109−buildenv() {
110− SEARX_DEBUG=1 pyenv.cmd python utils/build_env.py 2>&1 \
111− | prefix_stdout "${_Blue}BUILDENV${_creset} "
112− return "${PIPESTATUS[0]}"
113−}
114−
115−babel.compile() {
116− build_msg BABEL compile
117− pyenv.cmd pybabel compile -d "${REPO_ROOT}/searx/translations"
118− dump_return $?
119−}
120−
121−
122−data.all() {
123− data.languages
124− data.useragents
125− build_msg DATA "update searx/data/ahmia_blacklist.txt"
126− pyenv.cmd python searx_extra/update/update_ahmia_blacklist.py
127− build_msg DATA "update searx/data/wikidata_units.json"
128− pyenv.cmd python searx_extra/update/update_wikidata_units.py
129− build_msg DATA "update searx/data/currencies.json"
130− pyenv.cmd python searx_extra/update/update_currencies.py
131−}
132−
133−
134−data.languages() {
135− ( set -e
136− build_msg ENGINES "fetch languages .."
137− pyenv.cmd python searx_extra/update/update_languages.py
138− build_msg ENGINES "update update searx/languages.py"
139− build_msg DATA "update searx/data/engines_languages.json"
140− )
141− dump_return $?
142−}
143−
144−data.useragents() {
145− build_msg DATA "update searx/data/useragents.json"
146− pyenv.cmd python searx_extra/update/update_firefox_version.py
147− dump_return $?
148−}
149−
150−docs.prebuild() {
151− build_msg DOCS "build ${DOCS_BUILD}/includes"
152− (
153− set -e
154− [ "$VERBOSE" = "1" ] && set -x
155− mkdir -p "${DOCS_BUILD}/includes"
156− ./utils/searx.sh doc | cat > "${DOCS_BUILD}/includes/searx.rst"
157− ./utils/filtron.sh doc | cat > "${DOCS_BUILD}/includes/filtron.rst"
158− ./utils/morty.sh doc | cat > "${DOCS_BUILD}/includes/morty.rst"
159− )
160− dump_return $?
161−}
162−
163−docker.push() {
164− docker.build push
165−}
166−
167−# shellcheck disable=SC2119
168−docker.build() {
169− pyenv.install
170−
171− build_msg DOCKER build
172− # run installation in a subprocess and activate pyenv
173−
174− # See https://www.shellcheck.net/wiki/SC1001 and others ..
175− # shellcheck disable=SC2031,SC2230,SC2002,SC2236,SC2143,SC1001
176− ( set -e
177− # shellcheck source=/dev/null
178− source "${PY_ENV_BIN}/activate"
179−
180− # Check if it is a git repository
181− if [ ! -d .git ]; then
182− die 1 "This is not Git repository"
183− fi
184− if [ ! -x "$(which git)" ]; then
185− die 1 "git is not installed"
186− fi
187−
188− if ! git remote get-url origin 2> /dev/null; then
189− die 1 "there is no remote origin"
190− fi
191−
192− # This is a git repository
193−
194− # "git describe" to get the Docker version (for example : v0.15.0-89-g0585788e)
195− # awk to remove the "v" and the "g"
196− SEARX_GIT_VERSION=$(git describe --match "v[0-9]*\.[0-9]*\.[0-9]*" HEAD 2>/dev/null | awk -F'-' '{OFS="-"; $1=substr($1, 2); if ($3) { $3=substr($3, 2); } print}')
197−
198− # add the suffix "-dirty" if the repository has uncommited change
199− # /!\ HACK for searx/searx: ignore utils/brand.env
200− git update-index -q --refresh
201− if [ ! -z "$(git diff-index --name-only HEAD -- | grep -v 'utils/brand.env')" ]; then
202− SEARX_GIT_VERSION="${SEARX_GIT_VERSION}-dirty"
203− fi
204−
205− # Get the last git commit id, will be added to the Searx version (see Dockerfile)
206− VERSION_GITCOMMIT=$(echo "$SEARX_GIT_VERSION" | cut -d- -f2-4)
207− build_msg DOCKER "Last commit : $VERSION_GITCOMMIT"
208−
209− # Check consistency between the git tag and the searx/version.py file
210− # /! HACK : parse Python file with bash /!
211− # otherwise it is not possible build the docker image without all Python
212− # dependencies ( version.py loads __init__.py )
213− # SEARX_PYTHON_VERSION=$(python3 -c "import six; import searx.version; six.print_(searx.version.VERSION_STRING)")
214− SEARX_PYTHON_VERSION=$(cat searx/version.py | grep "\(VERSION_MAJOR\|VERSION_MINOR\|VERSION_BUILD\) =" | cut -d\= -f2 | sed -e 's/^[[:space:]]*//' | paste -sd "." -)
215− if [ "$(echo "$SEARX_GIT_VERSION" | cut -d- -f1)" != "$SEARX_PYTHON_VERSION" ]; then
216− err_msg "git tag: $SEARX_GIT_VERSION"
217− err_msg "searx/version.py: $SEARX_PYTHON_VERSION"
218− die 1 "Inconsistency between the last git tag and the searx/version.py file"
219− fi
220−
221− # define the docker image name
222− GITHUB_USER=$(echo "${GIT_URL}" | sed 's/.*github\.com\/\([^\/]*\).*/\1/')
223− SEARX_IMAGE_NAME="${SEARX_IMAGE_NAME:-${GITHUB_USER:-searx}/searx}"
224−
225− # build Docker image
226− build_msg DOCKER "Building image ${SEARX_IMAGE_NAME}:${SEARX_GIT_VERSION}"
227− docker build \
228− --build-arg GIT_URL="${GIT_URL}" \
229− --build-arg SEARX_GIT_VERSION="${SEARX_GIT_VERSION}" \
230− --build-arg VERSION_GITCOMMIT="${VERSION_GITCOMMIT}" \
231− --build-arg LABEL_DATE="$(date -u +"%Y-%m-%dT%H:%M:%SZ")" \
232− --build-arg LABEL_VCS_REF="$(git rev-parse HEAD)" \
233− --build-arg LABEL_VCS_URL="${GIT_URL}" \
234− --build-arg TIMESTAMP_SETTINGS="$(git log -1 --format="%cd" --date=unix -- searx/settings.yml)" \
235− --build-arg TIMESTAMP_UWSGI="$(git log -1 --format="%cd" --date=unix -- dockerfiles/uwsgi.ini)" \
236− -t "${SEARX_IMAGE_NAME}:latest" -t "${SEARX_IMAGE_NAME}:${SEARX_GIT_VERSION}" .
237−
238− if [ "$1" = "push" ]; then
239− docker push "${SEARX_IMAGE_NAME}:latest"
240− docker push "${SEARX_IMAGE_NAME}:${SEARX_GIT_VERSION}"
241− fi
242− )
243− dump_return $?
244−}
245−
246−# shellcheck disable=SC2119
247−gecko.driver() {
248− pyenv.install
249−
250− build_msg INSTALL "gecko.driver"
251− # run installation in a subprocess and activate pyenv
252− ( set -e
253− # shellcheck source=/dev/null
254− source "${PY_ENV_BIN}/activate"
255−
256− # TODO : check the current geckodriver version
257− geckodriver -V > /dev/null 2>&1 || NOTFOUND=1
258− set +e
259− if [ -z "$NOTFOUND" ]; then
260− build_msg INSTALL "geckodriver already installed"
261− return
262− fi
263− PLATFORM="$(python3 -c 'import platform; print(platform.system().lower(), platform.architecture()[0])')"
264− case "$PLATFORM" in
265− "linux 32bit" | "linux2 32bit") ARCH="linux32";;
266− "linux 64bit" | "linux2 64bit") ARCH="linux64";;
267− "windows 32 bit") ARCH="win32";;
268− "windows 64 bit") ARCH="win64";;
269− "mac 64bit") ARCH="macos";;
270− esac
271− GECKODRIVER_URL="https://github.com/mozilla/geckodriver/releases/download/$GECKODRIVER_VERSION/geckodriver-$GECKODRIVER_VERSION-$ARCH.tar.gz";
272−
273− build_msg GECKO "Installing ${PY_ENV_BIN}/geckodriver from $GECKODRIVER_URL"
274−
275− FILE="$(mktemp)"
276− wget -qO "$FILE" -- "$GECKODRIVER_URL" && tar xz -C "${PY_ENV_BIN}" -f "$FILE" geckodriver
277− rm -- "$FILE"
278− chmod 755 -- "${PY_ENV_BIN}/geckodriver"
279− )
280− dump_return $?
281−}
282−
283−node.env() {
284− local err=0
285− pushd "${REPO_ROOT}" &> /dev/null
286− # shellcheck disable=SC2230
287− which npm &> /dev/null || die 1 'node.env - npm is not found!'
288−
289− ( set -e
290− # shellcheck disable=SC2030
291− PATH="$(npm bin):$PATH"
292− export PATH
293−
294− build_msg INSTALL "npm install $NPM_PACKAGES"
295− # shellcheck disable=SC2086
296− npm install $NPM_PACKAGES
297−
298− cd "${REPO_ROOT}/searx/static/themes/oscar"
299− build_msg INSTALL "($(pwd)) npm install"
300− npm install
301−
302− build_msg INSTALL "($(pwd)) npm install"
303− cd "${REPO_ROOT}/searx/static/themes/simple"
304− npm install
305− )
306− err=$?
307− popd &> /dev/null
308− dump_return "$err"
309−}
310−
311−node.clean() {
312−
313− build_msg CLEAN "locally installed npm dependencies"
314− rm -rf \
315− ./node_modules \
316− ./package-lock.json \
317− ./searx/static/themes/oscar/package-lock.json \
318− ./searx/static/themes/oscar/node_modules \
319− ./searx/static/themes/simple/package-lock.json \
320− ./searx/static/themes/simple/node_modules
321− dump_return $?
322−}
323−
324−py.build() {
325− build_msg BUILD "[pylint] python package ${PYDIST}"
326− pyenv.cmd python setup.py \
327− sdist -d "${PYDIST}" \
328− bdist_wheel --bdist-dir "${PYBUILD}" -d "${PYDIST}"
329−}
330−
331−py.clean() {
332− build_msg CLEAN pyenv
333− ( set -e
334− pyenv.drop
335− [ "$VERBOSE" = "1" ] && set -x
336− rm -rf "${PYDIST}" "${PYBUILD}" "${PY_ENV}" ./.tox ./*.egg-info
337− find . -name '*.pyc' -exec rm -f {} +
338− find . -name '*.pyo' -exec rm -f {} +
339− find . -name __pycache__ -exec rm -rf {} +
340− )
341−}
342−
343−pyenv.check() {
344− cat <<EOF
345−import yaml
346−print('import yaml --> OK')
347−EOF
348−}
349−
350−pyenv.install() {
351−
352− if ! pyenv.OK; then
353− py.clean > /dev/null
354− fi
355− if pyenv.install.OK > /dev/null; then
356− return 0
357− fi
358− ( set -e
359− pyenv
360− build_msg PYENV "[install] pip install -e 'searx${PY_SETUP_EXTRAS}'"
361− "${PY_ENV_BIN}/python" -m pip install -e ".${PY_SETUP_EXTRAS}"
362− buildenv
363− ) || die 42 "error while build & install pyenv (${PY_ENV_BIN})"
364−}
365−
366−pyenv.uninstall() {
367− build_msg PYENV "[pyenv.uninstall] uninstall packages: ${PYOBJECTS}"
368− pyenv.cmd python setup.py develop --uninstall 2>&1 \
369− | prefix_stdout "${_Blue}PYENV ${_creset}[pyenv.uninstall] "
370−
371−}
372−
373−pypi.upload() {
374− py.clean
375− py.build
376− # https://github.com/pypa/twine
377− pyenv.cmd twine upload "${PYDIST}"/*
378−}
379−
380−pypi.upload.test() {
381− py.clean
382− py.build
383− pyenv.cmd twine upload -r testpypi "${PYDIST}"/*
384−}
385−
386−test.pylint() {
387− # shellcheck disable=SC2086
388− ( set -e
389− build_msg TEST "[pylint] \$PYLINT_FILES"
390− pyenv.cmd python ${PYLINT_OPTIONS} ${PYLINT_VERBOSE} \
391− "${PYLINT_FILES[@]}"
392−
393− build_msg TEST "[pylint] searx/engines"
394− pyenv.cmd python ${PYLINT_OPTIONS} ${PYLINT_VERBOSE} \
395− --disable="${PYLINT_SEARX_DISABLE_OPTION}" \
396− --additional-builtins="${PYLINT_ADDITIONAL_BUILTINS_FOR_ENGINES}" \
397− searx/engines
398−
399− build_msg TEST "[pylint] searx tests"
400− pyenv.cmd python ${PYLINT_OPTIONS} ${PYLINT_VERBOSE} \
401− --disable="${PYLINT_SEARX_DISABLE_OPTION}" \
402− --ignore=searx/engines \
403− tests
404− )
405− dump_return $?
406−}
407−
408−test.pep8() {
409− build_msg TEST 'pycodestyle (formerly pep8)'
410− local _exclude=""
411− printf -v _exclude '%s, ' "${PYLINT_FILES[@]}"
412− pyenv.cmd pycodestyle \
413− --exclude="searx/static, searx/languages.py, $_exclude " \
414− --max-line-length=120 \
415− --ignore "E117,E252,E402,E722,E741,W503,W504,W605" \
416− searx tests
417− dump_return $?
418−}
419−
420−test.unit() {
421− build_msg TEST 'tests/unit'
422− pyenv.cmd python -m nose2 -s tests/unit
423− dump_return $?
424−}
425−
426−test.coverage() {
427− build_msg TEST 'unit test coverage'
428− ( set -e
429− pyenv.cmd python -m nose2 -C --log-capture --with-coverage --coverage searx -s tests/unit
430− pyenv.cmd coverage report
431− pyenv.cmd coverage html
432− )
433− dump_return $?
434−}
435−
436−test.robot() {
437− build_msg TEST 'robot'
438− gecko.driver
439− PYTHONPATH=. pyenv.cmd python searx/testing.py robot
440− dump_return $?
441−}
442−
443−test.clean() {
444− build_msg CLEAN "test stuff"
445− rm -rf geckodriver.log .coverage coverage/
446− dump_return $?
447−}
448−
449−themes.all() {
450− ( set -e
451− node.env
452− themes.oscar
453− themes.simple
454− )
455− dump_return $?
456−}
457−
458−themes.oscar() {
459− local gruntfile=searx/static/themes/oscar/gruntfile.js
460− build_msg GRUNT "${gruntfile}"
461− PATH="$(npm bin):$PATH" grunt --gruntfile "${gruntfile}"
462− dump_return $?
463−}
464−
465−themes.simple() {
466− local gruntfile=searx/static/themes/simple/gruntfile.js
467− build_msg GRUNT "${gruntfile}"
468− PATH="$(npm bin):$PATH" grunt --gruntfile "${gruntfile}"
469− dump_return $?
470−}
471−
472−# shellcheck disable=SC2119
473−main() {
474−
475− local _type
476− local cmd="$1"; shift
477−
478− if [ "$cmd" == "" ]; then
479− help
480− err_msg "missing command"
481− return 42
482− fi
483−
484− case "$cmd" in
485− --getenv) var="$1"; echo "${!var}";;
486− --help) help;;
487− --*)
488− help
489− err_msg "unknown option $cmd"
490− return 42
491− ;;
492− *)
493− _type="$(type -t "$cmd")"
494− if [ "$_type" != 'function' ]; then
495− err_msg "unknown command $1 / use --help"
496− return 42
497− else
498− "$cmd" "$@"
499− fi
500− ;;
501− esac
502−}
503−
504−main "$@"
manage.shnew file+205
@@ -0,0 +1,205 @@
1+#!/bin/sh
2+
3+export LANG=C
4+
5+BASE_DIR="$(dirname -- "`readlink -f -- "$0"`")"
6+
7+cd -- "$BASE_DIR"
8+set -e
9+
10+# subshell
11+PYTHONPATH="$BASE_DIR"
12+SEARX_DIR="$BASE_DIR/searx"
13+ACTION="$1"
14+
15+. "${BASE_DIR}/utils/brand.env"
16+
17+#
18+# Python
19+#
20+
21+update_packages() {
22+ pip install --upgrade pip
23+ pip install --upgrade setuptools
24+ pip install -Ur "$BASE_DIR/requirements.txt"
25+}
26+
27+update_dev_packages() {
28+ update_packages
29+ pip install -Ur "$BASE_DIR/requirements-dev.txt"
30+}
31+
32+install_geckodriver() {
33+ echo '[!] Checking geckodriver'
34+ # TODO : check the current geckodriver version
35+ set -e
36+ geckodriver -V > /dev/null 2>&1 || NOTFOUND=1
37+ set +e
38+ if [ -z "$NOTFOUND" ]; then
39+ return
40+ fi
41+ GECKODRIVER_VERSION="v0.24.0"
42+ PLATFORM="`python3 -c "import platform; print(platform.system().lower(), platform.architecture()[0])"`"
43+ case "$PLATFORM" in
44+ "linux 32bit" | "linux2 32bit") ARCH="linux32";;
45+ "linux 64bit" | "linux2 64bit") ARCH="linux64";;
46+ "windows 32 bit") ARCH="win32";;
47+ "windows 64 bit") ARCH="win64";;
48+ "mac 64bit") ARCH="macos";;
49+ esac
50+ GECKODRIVER_URL="https://github.com/mozilla/geckodriver/releases/download/$GECKODRIVER_VERSION/geckodriver-$GECKODRIVER_VERSION-$ARCH.tar.gz";
51+
52+ if [ -z "$1" ]; then
53+ if [ -z "$VIRTUAL_ENV" ]; then
54+ printf "geckodriver can't be installed because VIRTUAL_ENV is not set, you should download it from\n %s" "$GECKODRIVER_URL"
55+ exit
56+ else
57+ GECKODRIVER_DIR="$VIRTUAL_ENV/bin"
58+ fi
59+ else
60+ GECKODRIVER_DIR="$1"
61+ mkdir -p -- "$GECKODRIVER_DIR"
62+ fi
63+
64+ printf "Installing %s/geckodriver from\n %s" "$GECKODRIVER_DIR" "$GECKODRIVER_URL"
65+
66+ FILE="`mktemp`"
67+ wget -qO "$FILE" -- "$GECKODRIVER_URL" && tar xz -C "$GECKODRIVER_DIR" -f "$FILE" geckodriver
68+ rm -- "$FILE"
69+ chmod 777 -- "$GECKODRIVER_DIR/geckodriver"
70+}
71+
72+locales() {
73+ pybabel compile -d "$SEARX_DIR/translations"
74+}
75+
76+
77+#
78+# Web
79+#
80+
81+npm_path_setup() {
82+ which npm || (printf 'Error: npm is not found\n'; exit 1)
83+ export PATH="$(npm bin)":$PATH
84+}
85+
86+npm_packages() {
87+ npm_path_setup
88+
89+ echo '[!] install NPM packages'
90+ cd -- "$BASE_DIR"
91+ npm install less@2.7 less-plugin-clean-css grunt-cli
92+
93+ echo '[!] install NPM packages for oscar theme'
94+ cd -- "$BASE_DIR/searx/static/themes/oscar"
95+ npm install
96+
97+ echo '[!] install NPM packages for simple theme'
98+ cd -- "$BASE_DIR/searx/static/themes/simple"
99+ npm install
100+}
101+
102+docker_build() {
103+ # Check if it is a git repository
104+ if [ ! -d .git ]; then
105+ echo "This is not Git repository"
106+ exit 1
107+ fi
108+
109+ if [ ! -x "$(which git)" ]; then
110+ echo "git is not installed"
111+ exit 1
112+ fi
113+
114+ if [ ! git remote get-url origin 2> /dev/null ]; then
115+ echo "there is no remote origin"
116+ exit 1
117+ fi
118+
119+ # This is a git repository
120+
121+ # "git describe" to get the Docker version (for example : v0.15.0-89-g0585788e)
122+ # awk to remove the "v" and the "g"
123+ SEARX_GIT_VERSION=$(git describe --match "v[0-9]*\.[0-9]*\.[0-9]*" HEAD 2>/dev/null | awk -F'-' '{OFS="-"; $1=substr($1, 2); if ($3) { $3=substr($3, 2); } print}')
124+
125+ # add the suffix "-dirty" if the repository has uncommited change
126+ # /!\ HACK for searx/searx: ignore searx/brand.py and utils/brand.env
127+ git update-index -q --refresh
128+ if [ ! -z "$(git diff-index --name-only HEAD -- | grep -v 'searx/brand.py' | grep -v 'utils/brand.env')" ]; then
129+ SEARX_GIT_VERSION="${SEARX_GIT_VERSION}-dirty"
130+ fi
131+
132+ # Get the last git commit id, will be added to the Searx version (see Dockerfile)
133+ VERSION_GITCOMMIT=$(echo $SEARX_GIT_VERSION | cut -d- -f2-4)
134+ echo "Last commit : $VERSION_GITCOMMIT"
135+
136+ # Check consistency between the git tag and the searx/version.py file
137+ # /!\ HACK : parse Python file with bash /!\
138+ # otherwise it is not possible build the docker image without all Python dependencies ( version.py loads __init__.py )
139+ # SEARX_PYTHON_VERSION=$(python3 -c "import six; import searx.version; six.print_(searx.version.VERSION_STRING)")
140+ SEARX_PYTHON_VERSION=$(cat searx/version.py | grep "\(VERSION_MAJOR\|VERSION_MINOR\|VERSION_BUILD\) =" | cut -d\= -f2 | sed -e 's/^[[:space:]]*//' | paste -sd "." -)
141+ if [ $(echo "$SEARX_GIT_VERSION" | cut -d- -f1) != "$SEARX_PYTHON_VERSION" ]; then
142+ echo "Inconsistency between the last git tag and the searx/version.py file"
143+ echo "git tag: $SEARX_GIT_VERSION"
144+ echo "searx/version.py: $SEARX_PYTHON_VERSION"
145+ exit 1
146+ fi
147+
148+ # define the docker image name
149+ GITHUB_USER=$(echo "${GIT_URL}" | sed 's/.*github\.com\/\([^\/]*\).*/\1/')
150+ SEARX_IMAGE_NAME="${GITHUB_USER:-searx}/searx"
151+
152+ # build Docker image
153+ echo "Building image ${SEARX_IMAGE_NAME}:${SEARX_GIT_VERSION}"
154+ sudo docker build \
155+ --build-arg GIT_URL="${GIT_URL}" \
156+ --build-arg SEARX_GIT_VERSION="${SEARX_GIT_VERSION}" \
157+ --build-arg VERSION_GITCOMMIT="${VERSION_GITCOMMIT}" \
158+ --build-arg LABEL_DATE=$(date -u +"%Y-%m-%dT%H:%M:%SZ") \
159+ --build-arg LABEL_VCS_REF=$(git rev-parse HEAD) \
160+ --build-arg LABEL_VCS_URL="${GIT_URL}" \
161+ --build-arg TIMESTAMP_SETTINGS=$(git log -1 --format="%cd" --date=unix -- searx/settings.yml) \
162+ --build-arg TIMESTAMP_UWSGI=$(git log -1 --format="%cd" --date=unix -- dockerfiles/uwsgi.ini) \
163+ -t ${SEARX_IMAGE_NAME}:latest -t ${SEARX_IMAGE_NAME}:${SEARX_GIT_VERSION} .
164+
165+ if [ "$1" = "push" ]; then
166+ sudo docker push ${SEARX_IMAGE_NAME}:latest
167+ sudo docker push ${SEARX_IMAGE_NAME}:${SEARX_GIT_VERSION}
168+ fi
169+}
170+
171+#
172+# Help
173+#
174+
175+help() {
176+ [ -z "$1" ] || printf 'Error: %s\n' "$1"
177+ echo "Searx manage.sh help
178+
179+Commands
180+========
181+ help - This text
182+
183+ Build requirements
184+ ------------------
185+ update_packages - Check & update production dependency changes
186+ update_dev_packages - Check & update development and production dependency changes
187+ install_geckodriver - Download & install geckodriver if not already installed (required for robot_tests)
188+ npm_packages - Download & install npm dependencies
189+
190+ Build
191+ -----
192+ locales - Compile locales
193+
194+Environment:
195+ GIT_URL: ${GIT_URL}
196+ ISSUE_URL: ${ISSUE_URL}
197+ SEARX_URL: ${SEARX_URL}
198+ DOCS_URL: ${DOCS_URL}
199+ PUBLIC_INSTANCES: ${PUBLIC_INSTANCES}
200+"
201+}
202+
203+[ "$(command -V "$ACTION" | grep ' function$')" = "" ] \
204+ && help "action not found" \
205+ || "$ACTION" "$2"
requirements-dev.txt+23−19
@@ -1,20 +1,24 @@
1−mock==4.0.3
2−nose2[coverage_plugin]==0.10.0
1+mock==2.0.0
2+nose2[coverage_plugin]==0.9.2
33cov-core==1.15.0
4−pycodestyle==2.8.0
5−pylint==2.12.2
6−splinter==0.17.0
7−transifex-client==0.14.3; python_version < '3.10'
8−transifex-client==0.12.1; python_version == '3.10'
9−selenium==4.1.0
10−twine==3.7.1
11−Pallets-Sphinx-Themes==2.0.2
12−docutils==0.16
13−Sphinx==4.4.0
14−sphinx-issues==3.0.1
15−sphinx-jinja==1.4.0
16−sphinx-tabs==3.2.0
17−sphinxcontrib-programoutput==0.17
18−sphinx-autobuild==2021.3.14
19−linuxdoc==20211220
20−aiounittest==1.4.1
4+pycodestyle==2.6.0
5+pylint==2.4.4
6+plone.testing==5.0.0
7+splinter==0.11.0
8+transifex-client==0.12.2
9+unittest2==1.1.0
10+zope.testrunner==4.5.1
11+selenium==3.141.0
12+twine==3.2.0; python_version >= "3.6"
13+twine==1.15.0; python_version < "3.6"
14+Pallets-Sphinx-Themes==1.2.3
15+Sphinx==3.2.1; python_version >= '3.6'
16+Sphinx==3.0.1; python_version < '3.6'
17+sphinx-issues==1.2.0
18+sphinx-jinja==1.1.1
19+sphinx-tabs==1.3.0; python_version >= '3.6'
20+sphinx-tabs==1.1.13; python_version < '3.6'
21+sphinxcontrib-programoutput==0.16
22+sphinx-autobuild==2020.9.1; python_version >= '3.6'
23+sphinx-autobuild==0.7.1; python_version < '3.6'
24+linuxdoc @ git+http://github.com/return42/linuxdoc.git@70673dcf69e705e08d81f53794895dc15c4920b3#egg=linuxdoc
requirements.txt+12−16
@@ -1,16 +1,12 @@
1−certifi==2021.10.8
2−babel==2.9.1
3−flask-babel==2.0.0
4−flask==2.0.2
5−jinja2==3.0.3
6−lxml==4.7.1
7−pygments==2.8.0
8−python-dateutil==2.8.2
9−pyyaml==6.0
10−httpx[http2]==0.21.3
11−Brotli==1.0.9
12−uvloop==0.16.0; python_version >= '3.7'
13−uvloop==0.14.0; python_version < '3.7'
14−httpx-socks[asyncio]==0.7.2
15−langdetect==1.0.9
16−setproctitle==1.2.2
1+certifi==2020.6.20
2+babel==2.7.0
3+flask-babel==1.0.0
4+flask==1.1.2
5+idna==2.10
6+jinja2==2.11.1
7+lxml==4.5.0
8+pygments==2.1.3
9+pyopenssl==19.1.0
10+python-dateutil==2.8.0
11+pyyaml==5.3.1
12+requests[socks]==2.24.0
searx/__init__.py+31−48
@@ -16,15 +16,39 @@ along with searx. If not, see < http://www.gnu.org/licenses/ >.
1616'''
1717
1818import logging
19−import searx.settings_loader
2019from os import environ
2120from os.path import realpath, dirname, join, abspath, isfile
21+from io import open
22+from yaml import safe_load
2223
2324
2425searx_dir = abspath(dirname(__file__))
2526engine_dir = dirname(realpath(__file__))
2627static_path = abspath(join(dirname(__file__), 'static'))
27−settings, settings_load_message = searx.settings_loader.load_settings()
28+
29+
30+def check_settings_yml(file_name):
31+ if isfile(file_name):
32+ return file_name
33+ else:
34+ return None
35+
36+
37+# find location of settings.yml
38+if 'SEARX_SETTINGS_PATH' in environ:
39+ # if possible set path to settings using the
40+ # enviroment variable SEARX_SETTINGS_PATH
41+ settings_path = check_settings_yml(environ['SEARX_SETTINGS_PATH'])
42+else:
43+ # if not, get it from searx code base or last solution from /etc/searx
44+ settings_path = check_settings_yml(join(searx_dir, 'settings.yml')) or check_settings_yml('/etc/searx/settings.yml')
45+
46+if not settings_path:
47+ raise Exception('settings.yml not found')
48+
49+# load settings
50+with open(settings_path, 'r', encoding='utf-8') as settings_yaml:
51+ settings = safe_load(settings_yaml)
2852
2953if settings['ui']['static_path']:
3054 static_path = settings['ui']['static_path']
@@ -34,6 +58,7 @@ enable debug if
3458the environnement variable SEARX_DEBUG is 1 or true
3559(whatever the value in settings.yml)
3660or general.debug=True in settings.yml
61+
3762disable debug if
3863the environnement variable SEARX_DEBUG is 0 or false
3964(whatever the value in settings.yml)
@@ -53,7 +78,7 @@ else:
5378 logging.basicConfig(level=logging.WARNING)
5479
5580logger = logging.getLogger('searx')
56−logger.info(settings_load_message)
81+logger.debug('read configuration from %s', settings_path)
5782logger.info('Initialisation done')
5883
5984if 'SEARX_SECRET' in environ:
@@ -61,48 +86,6 @@ if 'SEARX_SECRET' in environ:
6186if 'SEARX_BIND_ADDRESS' in environ:
6287 settings['server']['bind_address'] = environ['SEARX_BIND_ADDRESS']
6388
64−
65−class _brand_namespace:
66−
67− @classmethod
68− def get_val(cls, group, name, default=''):
69− return settings.get(group, {}).get(name) or default
70−
71− @property
72− def SEARX_URL(self):
73− return self.get_val('server', 'base_url')
74−
75− @property
76− def CONTACT_URL(self):
77− return self.get_val('general', 'contact_url')
78−
79− @property
80− def GIT_URL(self):
81− return self.get_val('brand', 'git_url')
82−
83− @property
84− def GIT_BRANCH(self):
85− return self.get_val('brand', 'git_branch')
86−
87− @property
88− def ISSUE_URL(self):
89− return self.get_val('brand', 'issue_url')
90−
91− @property
92− def DOCS_URL(self):
93− return self.get_val('brand', 'docs_url')
94−
95− @property
96− def PUBLIC_INSTANCES(self):
97− return self.get_val('brand', 'public_instances')
98−
99− @property
100− def WIKI_URL(self):
101− return self.get_val('brand', 'wiki_url')
102−
103− @property
104− def TWITTER_URL(self):
105− return self.get_val('brand', 'twitter_url')
106−
107−
108−brand = _brand_namespace()
89+if not searx_debug and settings['server']['secret_key'] == 'ultrasecretkey':
90+ logger.error('server.secret_key is not changed. Please use something else instead of ultrasecretkey.')
91+ exit(1)
searx/answerers/__init__.py+1−1
@@ -32,7 +32,7 @@ def ask(query):
3232 results = []
3333 query_parts = list(filter(None, query.query.split()))
3434
35− if not query_parts or query_parts[0] not in answerers_by_keywords:
35+ if query_parts[0] not in answerers_by_keywords:
3636 return results
3737
3838 for answerer in answerers_by_keywords[query_parts[0]]:
searx/autocomplete.py+87−38
@@ -20,41 +20,100 @@ from lxml import etree
2020from json import loads
2121from urllib.parse import urlencode
2222
23−from httpx import HTTPError
24−
25−
2623from searx import settings
27−from searx.data import ENGINES_LANGUAGES
28−from searx.network import get as http_get
29−from searx.exceptions import SearxEngineResponseException
24+from searx.languages import language_codes
25+from searx.engines import (
26+ categories, engines, engine_shortcuts
27+)
28+from searx.poolrequests import get as http_get
3029
3130
3231def get(*args, **kwargs):
3332 if 'timeout' not in kwargs:
3433 kwargs['timeout'] = settings['outgoing']['request_timeout']
35− kwargs['raise_for_httperror'] = True
36− return http_get(*args, **kwargs)
3734
35+ return http_get(*args, **kwargs)
3836
39−def brave(query, lang):
40− # brave search autocompleter
41− url = 'https://search.brave.com/api/suggest?{query}'
4237
43− resp = get(url.format(query=urlencode({'q': query})))
38+def searx_bang(full_query):
39+ '''check if the searchQuery contain a bang, and create fitting autocompleter results'''
40+ # check if there is a query which can be parsed
41+ if len(full_query.getQuery()) == 0:
42+ return []
4443
4544 results = []
4645
47− if resp.ok:
48− data = loads(resp.text)
49− for suggestion in data[1]:
50− results.append(suggestion)
51−
52− return results
46+ # check if current query stats with !bang
47+ first_char = full_query.getQuery()[0]
48+ if first_char == '!' or first_char == '?':
49+ if len(full_query.getQuery()) == 1:
50+ # show some example queries
51+ # TODO, check if engine is not avaliable
52+ results.append(first_char + "images")
53+ results.append(first_char + "wikipedia")
54+ results.append(first_char + "osm")
55+ else:
56+ engine_query = full_query.getQuery()[1:]
57+
58+ # check if query starts with categorie name
59+ for categorie in categories:
60+ if categorie.startswith(engine_query):
61+ results.append(first_char + '{categorie}'.format(categorie=categorie))
62+
63+ # check if query starts with engine name
64+ for engine in engines:
65+ if engine.startswith(engine_query.replace('_', ' ')):
66+ results.append(first_char + '{engine}'.format(engine=engine.replace(' ', '_')))
67+
68+ # check if query starts with engine shortcut
69+ for engine_shortcut in engine_shortcuts:
70+ if engine_shortcut.startswith(engine_query):
71+ results.append(first_char + '{engine_shortcut}'.format(engine_shortcut=engine_shortcut))
72+
73+ # check if current query stats with :bang
74+ elif first_char == ':':
75+ if len(full_query.getQuery()) == 1:
76+ # show some example queries
77+ results.append(":en")
78+ results.append(":en_us")
79+ results.append(":english")
80+ results.append(":united_kingdom")
81+ else:
82+ engine_query = full_query.getQuery()[1:]
83+
84+ for lc in language_codes:
85+ lang_id, lang_name, country, english_name = map(str.lower, lc)
86+
87+ # check if query starts with language-id
88+ if lang_id.startswith(engine_query):
89+ if len(engine_query) <= 2:
90+ results.append(':{lang_id}'.format(lang_id=lang_id.split('-')[0]))
91+ else:
92+ results.append(':{lang_id}'.format(lang_id=lang_id))
93+
94+ # check if query starts with language name
95+ if lang_name.startswith(engine_query) or english_name.startswith(engine_query):
96+ results.append(':{lang_name}'.format(lang_name=lang_name))
97+
98+ # check if query starts with country
99+ if country.startswith(engine_query.replace('_', ' ')):
100+ results.append(':{country}'.format(country=country.replace(' ', '_')))
101+
102+ # remove duplicates
103+ result_set = set(results)
104+
105+ # remove results which are already contained in the query
106+ for query_part in full_query.query_parts:
107+ if query_part in result_set:
108+ result_set.remove(query_part)
109+
110+ # convert result_set back to list
111+ return list(result_set)
53112
54113
55114def dbpedia(query, lang):
56115 # dbpedia autocompleter, no HTTPS
57− autocomplete_url = 'https://lookup.dbpedia.org/api/search.asmx/KeywordSearch?'
116+ autocomplete_url = 'http://lookup.dbpedia.org/api/search.asmx/KeywordSearch?'
58117
59118 response = get(autocomplete_url + urlencode(dict(QueryString=query)))
60119
@@ -62,7 +121,8 @@ def dbpedia(query, lang):
62121
63122 if response.ok:
64123 dom = etree.fromstring(response.content)
65− results = dom.xpath('//Result/Label//text()')
124+ results = dom.xpath('//a:Result/a:Label//text()',
125+ namespaces={'a': 'http://lookup.dbpedia.org/'})
66126
67127 return results
68128
@@ -94,11 +154,12 @@ def google(query, lang):
94154
95155def startpage(query, lang):
96156 # startpage autocompleter
97− lui = ENGINES_LANGUAGES['startpage'].get(lang, 'english')
98− url = 'https://startpage.com/suggestions?{query}'
99− resp = get(url.format(query=urlencode({'q': query, 'segment': 'startpage.udog', 'lui': lui})))
100− data = resp.json()
101− return [e['text'] for e in data.get('suggestions', []) if 'text' in e]
157+ url = 'https://startpage.com/do/suggest?{query}'
158+
159+ resp = get(url.format(query=urlencode({'query': query}))).text.split('\n')
160+ if len(resp) > 1:
161+ return resp
162+ return []
102163
103164
104165def swisscows(query, lang):
@@ -136,8 +197,7 @@ def wikipedia(query, lang):
136197 return []
137198
138199
139−backends = {'brave': brave,
140− 'dbpedia': dbpedia,
200+backends = {'dbpedia': dbpedia,
141201 'duckduckgo': duckduckgo,
142202 'google': google,
143203 'startpage': startpage,
@@ -145,14 +205,3 @@ backends = {'brave': brave,
145205 'qwant': qwant,
146206 'wikipedia': wikipedia
147207 }
148−
149−
150−def search_autocomplete(backend_name, query, lang):
151− backend = backends.get(backend_name)
152− if backend is None:
153− return []
154−
155− try:
156− return backend(query, lang)
157− except (HTTPError, SearxEngineResponseException):
158− return []
searx/brand.pynew file+6
@@ -0,0 +1,6 @@
1+GIT_URL = 'https://github.com/searx/searx'
2+GIT_BRANCH = 'master'
3+ISSUE_URL = 'https://github.com/searx/searx/issues'
4+SEARX_URL = 'https://searx.me'
5+DOCS_URL = 'https://searx.github.io/searx'
6+PUBLIC_INSTANCES = 'https://searx.space'
searx/data/__init__.py+17−38
@@ -1,50 +1,29 @@
1−# SPDX-License-Identifier: AGPL-3.0-or-later
2−# lint: pylint
3−"""This module holds the *data* created by::
4−
5− make data.all
6−
7−"""
8−
9−__all__ = [
10− 'ENGINES_LANGUAGES',
11− 'CURRENCIES',
12− 'USER_AGENTS',
13− 'EXTERNAL_URLS',
14− 'WIKIDATA_UNITS',
15− 'EXTERNAL_BANGS',
16− 'OSM_KEYS_TAGS',
17− 'ahmia_blacklist_loader',
18−]
19−
201import json
212from pathlib import Path
223
23−data_dir = Path(__file__).parent
244
5+__init__ = ['ENGINES_LANGUGAGES', 'CURRENCIES', 'USER_AGENTS', 'EXTERNAL_URLS', 'WIKIDATA_UNITS',
6+ 'bangs_loader', 'ahmia_blacklist_loader']
7+data_dir = Path(__file__).parent
258
26−def _load(filename):
27− with open(data_dir / filename, encoding='utf-8') as f:
28− return json.load(f)
299
10+def load(filename):
11+ # add str(...) for Python 3.5
12+ with open(str(data_dir / filename), encoding='utf-8') as fd:
13+ return json.load(fd)
3014
31−def ahmia_blacklist_loader():
32− """Load data from `ahmia_blacklist.txt` and return a list of MD5 values of onion
33− names. The MD5 values are fetched by::
3415
35− searx_extra/update/update_ahmia_blacklist.py
16+def bangs_loader():
17+ return load('bangs.json')
3618
37− This function is used by :py:mod:`searx.plugins.ahmia_filter`.
3819
39− """
40− with open(str(data_dir / 'ahmia_blacklist.txt'), encoding='utf-8') as f:
41− return f.read().split()
20+def ahmia_blacklist_loader():
21+ with open(str(data_dir / 'ahmia_blacklist.txt'), encoding='utf-8') as fd:
22+ return fd.read().split()
4223
4324
44−ENGINES_LANGUAGES = _load('engines_languages.json')
45−CURRENCIES = _load('currencies.json')
46−USER_AGENTS = _load('useragents.json')
47−EXTERNAL_URLS = _load('external_urls.json')
48−WIKIDATA_UNITS = _load('wikidata_units.json')
49−EXTERNAL_BANGS = _load('external_bangs.json')
50−OSM_KEYS_TAGS = _load('osm_keys_tags.json')
25+ENGINES_LANGUAGES = load('engines_languages.json')
26+CURRENCIES = load('currencies.json')
27+USER_AGENTS = load('useragents.json')
28+EXTERNAL_URLS = load('external_urls.json')
29+WIKIDATA_UNITS = load('wikidata_units.json')
searx/data/ahmia_blacklist.txt−19553
This diff is too big to show here.
searx/data/bangs.jsonnew file+67913
This diff is too big to show here.
searx/data/currencies.json+7619−14643
This diff is too big to show here.
searx/data/engines_languages.json+1486−656
This diff is too big to show here.
searx/data/external_bangs.jsondeleted−19068
This diff is too big to show here.
searx/data/osm_keys_tags.jsondeleted−15416
This diff is too big to show here.
searx/data/useragents.json+4−5
This diff is too big to show here.
searx/data/wikidata_units.json+58−275
This diff is too big to show here.
searx/engines/1337x.py+8−22
This diff is too big to show here.
searx/engines/__init__.py+22−42
This diff is too big to show here.
searx/engines/acgsou.pynew file+74
This diff is too big to show here.
searx/engines/ahmia.py+16−19
This diff is too big to show here.
searx/engines/apkmirror.py+25−31
This diff is too big to show here.
searx/engines/archlinux.py+12−15
This diff is too big to show here.
searx/engines/arxiv.py+24−22
This diff is too big to show here.
searx/engines/bandcamp.pydeleted−73
This diff is too big to show here.
searx/engines/base.py+15−11
This diff is too big to show here.
searx/engines/bing.py+15−13
This diff is too big to show here.
searx/engines/bing_images.py+14−14
This diff is too big to show here.
searx/engines/bing_news.py+17−20
This diff is too big to show here.
searx/engines/bing_videos.py+10−13
This diff is too big to show here.
searx/engines/btdigg.py+9−14
This diff is too big to show here.
searx/engines/ccengine.pydeleted−53
This diff is too big to show here.
searx/engines/command.py+19−7
This diff is too big to show here.
searx/engines/core.pydeleted−82
This diff is too big to show here.
searx/engines/currency_convert.py+46−18
This diff is too big to show here.
searx/engines/dailymotion.py+11−11
This diff is too big to show here.
searx/engines/deezer.py+8−11
This diff is too big to show here.
searx/engines/deviantart.py+54−49
This diff is too big to show here.
searx/engines/dictzone.py+27−19
This diff is too big to show here.
searx/engines/digbt.py+8−10
This diff is too big to show here.
searx/engines/digg.py+44−44
This diff is too big to show here.
searx/engines/doku.py+11−14
This diff is too big to show here.
searx/engines/duckduckgo.py+57−112
This diff is too big to show here.
searx/engines/duckduckgo_definitions.py+11−17
This diff is too big to show here.
searx/engines/duckduckgo_images.py+23−20
This diff is too big to show here.
searx/engines/duden.py+32−30
This diff is too big to show here.
searx/engines/dummy-offline.py+3−11
This diff is too big to show here.
searx/engines/dummy.py+3−11
This diff is too big to show here.
searx/engines/ebay.py+9−14
This diff is too big to show here.
searx/engines/elasticsearch.pydeleted−144
This diff is too big to show here.
searx/engines/etools.py+8−11
This diff is too big to show here.
searx/engines/fdroid.py+7−11
This diff is too big to show here.
searx/engines/filecrop.pynew file+85
This diff is too big to show here.
searx/engines/flickr.py+9−11
This diff is too big to show here.
searx/engines/flickr_noapi.py+11−12
This diff is too big to show here.
searx/engines/framalibre.py+8−11
This diff is too big to show here.
searx/engines/freesound.pydeleted−69
This diff is too big to show here.
searx/engines/frinkiac.py+8−15
This diff is too big to show here.
searx/engines/genius.py+17−18
This diff is too big to show here.
searx/engines/gentoo.py+10−11
This diff is too big to show here.
searx/engines/gigablast.py+23−35
This diff is too big to show here.
searx/engines/github.py+9−12
This diff is too big to show here.
searx/engines/google.py+94−139
This diff is too big to show here.
searx/engines/google_images.py+95−80
This diff is too big to show here.
searx/engines/google_news.py+57−167
This diff is too big to show here.
searx/engines/google_scholar.pydeleted−156
This diff is too big to show here.
searx/engines/google_videos.py+69−173
This diff is too big to show here.
searx/engines/imdb.pydeleted−55
This diff is too big to show here.
searx/engines/ina.py+18−16
This diff is too big to show here.
searx/engines/invidious.py+14−29
This diff is too big to show here.
searx/engines/json_engine.py+5−17
This diff is too big to show here.
searx/engines/kickass.py+8−11
This diff is too big to show here.
searx/engines/loc.pydeleted−68
This diff is too big to show here.
searx/engines/mediathekviewweb.pydeleted−68
This diff is too big to show here.
searx/engines/mediawiki.py+12−12
This diff is too big to show here.
searx/engines/meilisearch.pydeleted−59
This diff is too big to show here.
searx/engines/microsoft_academic.py+40−37
This diff is too big to show here.
searx/engines/mixcloud.py+8−11
This diff is too big to show here.
searx/engines/mongodb.pydeleted−61
This diff is too big to show here.
searx/engines/mysql_server.pydeleted−62
This diff is too big to show here.
searx/engines/not_evil.pynew file+64
This diff is too big to show here.
searx/engines/nyaa.py+7−11
This diff is too big to show here.
searx/engines/onesearch.pydeleted−57
This diff is too big to show here.
searx/engines/opensemantic.py+9−13
This diff is too big to show here.
searx/engines/openstreetmap.py+74−407
This diff is too big to show here.
searx/engines/pdbe.py+8−11
This diff is too big to show here.
searx/engines/peertube.py+20−21
This diff is too big to show here.
searx/engines/photon.py+10−12
This diff is too big to show here.
searx/engines/piratebay.py+11−16
This diff is too big to show here.
searx/engines/postgresql.pydeleted−69
This diff is too big to show here.
searx/engines/prowlarr.pydeleted−80
This diff is too big to show here.
searx/engines/pubmed.py+12−16
This diff is too big to show here.
searx/engines/qwant.py+87−199
This diff is too big to show here.
searx/engines/recoll.pydeleted−111
This diff is too big to show here.
searx/engines/reddit.py+8−11
This diff is too big to show here.
searx/engines/redis_server.pydeleted−69
This diff is too big to show here.
searx/engines/rumble.pydeleted−81
This diff is too big to show here.
searx/engines/scanr_structures.py+8−11
This diff is too big to show here.
searx/engines/searchcode_code.py+9−11
This diff is too big to show here.
searx/engines/searchcode_doc.pynew file+49
This diff is too big to show here.
searx/engines/searx_engine.py+8−10
This diff is too big to show here.
searx/engines/seedpeer.pynew file+78
This diff is too big to show here.
searx/engines/semantic_scholar.pydeleted−42
This diff is too big to show here.
searx/engines/sepiasearch.py+9−14
This diff is too big to show here.
searx/engines/seznam.pydeleted−66
This diff is too big to show here.
searx/engines/sjp.pydeleted−92
This diff is too big to show here.
searx/engines/solidtorrents.pydeleted−53
This diff is too big to show here.
searx/engines/solr.pydeleted−74
This diff is too big to show here.
searx/engines/soundcloud.py+13−14
This diff is too big to show here.
searx/engines/spotify.py+10−14
This diff is too big to show here.
searx/engines/springer.pydeleted−74
This diff is too big to show here.
searx/engines/sqlite.pydeleted−51
This diff is too big to show here.
searx/engines/stackexchange.pydeleted−67
This diff is too big to show here.
searx/engines/stackoverflow.pynew file+57
This diff is too big to show here.
searx/engines/startpage.py+33−114
This diff is too big to show here.
searx/engines/tineye.pydeleted−86
This diff is too big to show here.
searx/engines/tokyotoshokan.py+8−11
This diff is too big to show here.
searx/engines/torrentz.py+9−11
This diff is too big to show here.
searx/engines/translated.py+29−16
This diff is too big to show here.
searx/engines/twitter.pynew file+87
This diff is too big to show here.
searx/engines/unsplash.py+8−11
This diff is too big to show here.
searx/engines/vimeo.py+13−14
This diff is too big to show here.
searx/engines/wikidata.py+17−23
This diff is too big to show here.
searx/engines/wikipedia.py+12−35
This diff is too big to show here.
searx/engines/wolframalpha_api.py+9−14
This diff is too big to show here.
searx/engines/wolframalpha_noapi.py+10−15
This diff is too big to show here.
searx/engines/wordnik.pydeleted−77
This diff is too big to show here.
searx/engines/www1x.py+21−20
This diff is too big to show here.
searx/engines/xpath.py+15−20
This diff is too big to show here.
searx/engines/xpath_flex.pydeleted−121
This diff is too big to show here.
searx/engines/yacy.py+16−16
This diff is too big to show here.
searx/engines/yahoo.py+101−104
This diff is too big to show here.
searx/engines/yahoo_news.py+82−84
This diff is too big to show here.
searx/engines/yandex.pynew file+64
This diff is too big to show here.
searx/engines/yggtorrent.py+13−18
This diff is too big to show here.
searx/engines/youtube_api.py+11−16
This diff is too big to show here.
searx/engines/youtube_noapi.py+18−85
This diff is too big to show here.
searx/exceptions.py−70
This diff is too big to show here.
searx/external_bang.py+26−76
This diff is too big to show here.
searx/external_urls.py+2−14
This diff is too big to show here.
searx/languages.py+3−1
This diff is too big to show here.
searx/metrology/__init__.pydeleted
No changes to the content.
searx/metrology/error_recorder.pydeleted−144
This diff is too big to show here.
searx/network/__init__.pydeleted−188
This diff is too big to show here.
searx/network/client.pydeleted−219
This diff is too big to show here.
searx/network/network.pydeleted−358
This diff is too big to show here.
searx/network/raise_for_httperror.pydeleted−66
This diff is too big to show here.
searx/plugins/__init__.py+3−4
This diff is too big to show here.
searx/plugins/hostname_replace.pydeleted−28
This diff is too big to show here.
searx/plugins/oa_doi_rewrite.py+2−2
This diff is too big to show here.
searx/plugins/self_info.py+1−1
This diff is too big to show here.
searx/plugins/tracker_url_remover.py−1
This diff is too big to show here.
searx/poolrequests.pynew file+184
This diff is too big to show here.
searx/preferences.py+8−19
This diff is too big to show here.
searx/query.py+118−286
This diff is too big to show here.
searx/results.py+9−36
This diff is too big to show here.
searx/search.pynew file+467
This diff is too big to show here.
searx/search/__init__.pydeleted−216
This diff is too big to show here.
searx/search/checker/__init__.pydeleted−4
This diff is too big to show here.
searx/search/checker/__main__.pydeleted−95
This diff is too big to show here.
searx/search/checker/background.pydeleted−125
This diff is too big to show here.
searx/search/checker/impl.pydeleted−416
This diff is too big to show here.
searx/search/models.pydeleted−71
This diff is too big to show here.
searx/search/processors/__init__.pydeleted−41
This diff is too big to show here.
searx/search/processors/abstract.pydeleted−52
This diff is too big to show here.
searx/search/processors/offline.pydeleted−51
This diff is too big to show here.
searx/search/processors/online.pydeleted−262
This diff is too big to show here.
searx/search/processors/online_currency.pydeleted−69
This diff is too big to show here.
searx/search/processors/online_dictionary.pydeleted−55
This diff is too big to show here.
searx/settings.yml+159−858
This diff is too big to show here.
searx/settings_loader.pydeleted−132
This diff is too big to show here.
searx/settings_robot.yml+2−8
This diff is too big to show here.
searx/shared/__init__.pydeleted−31
This diff is too big to show here.
searx/shared/shared_abstract.pydeleted−21
This diff is too big to show here.
searx/shared/shared_simple.pydeleted−39
This diff is too big to show here.
searx/shared/shared_uwsgi.pydeleted−64
This diff is too big to show here.
searx/static/css/bootstrap.min.cssnew file+1
This diff is too big to show here.
searx/static/fonts/glyphicons-halflings-regular.eotnew file
Binary file not shown.
searx/static/fonts/glyphicons-halflings-regular.svgnew file+229
This diff is too big to show here.
searx/static/fonts/glyphicons-halflings-regular.ttfnew file
Binary file not shown.
searx/static/fonts/glyphicons-halflings-regular.woffnew file
Binary file not shown.
searx/static/js/bootstrap.min.jsnew file+6
This diff is too big to show here.
searx/static/js/html5shiv.min.jsnew file+4
This diff is too big to show here.
searx/static/js/jquery-1.11.1.min.jsnew file+4
This diff is too big to show here.
searx/static/js/mootools-autocompleter-1.1.2-min.jsnew file+2
This diff is too big to show here.
searx/static/js/mootools-core-1.4.5-min.jsnew file+491
This diff is too big to show here.
searx/static/js/require-2.1.15.min.jsnew file+36
This diff is too big to show here.
searx/static/js/respond.min.jsnew file+5
This diff is too big to show here.
searx/static/js/typeahead.bundle.min.jsnew file+7
This diff is too big to show here.
searx/static/less/bootstrap/.csscomb.jsonnew file+297
This diff is too big to show here.
searx/static/less/bootstrap/.csslintrcnew file+19
This diff is too big to show here.
searx/static/less/bootstrap/alerts.lessnew file+68
This diff is too big to show here.
searx/static/less/bootstrap/badges.lessnew file+55
This diff is too big to show here.
searx/static/less/bootstrap/bootstrap.lessnew file+53
This diff is too big to show here.
searx/static/less/bootstrap/breadcrumbs.lessnew file+26
This diff is too big to show here.
searx/static/less/bootstrap/button-groups.lessnew file+240
This diff is too big to show here.
searx/static/less/bootstrap/buttons.lessnew file+157
This diff is too big to show here.
searx/static/less/bootstrap/carousel.lessnew file+243
This diff is too big to show here.
searx/static/less/bootstrap/close.lessnew file+33
This diff is too big to show here.
searx/static/less/bootstrap/code.lessnew file+68
This diff is too big to show here.
searx/static/less/bootstrap/component-animations.lessnew file+31
This diff is too big to show here.
searx/static/less/bootstrap/dropdowns.lessnew file+215
This diff is too big to show here.
searx/static/less/bootstrap/forms.lessnew file+540
This diff is too big to show here.
searx/static/less/bootstrap/glyphicons.lessnew file+233
This diff is too big to show here.
searx/static/less/bootstrap/grid.lessnew file+84
This diff is too big to show here.
searx/static/less/bootstrap/input-groups.lessnew file+166
This diff is too big to show here.
searx/static/less/bootstrap/jumbotron.lessnew file+48
This diff is too big to show here.
searx/static/less/bootstrap/labels.lessnew file+64
This diff is too big to show here.
searx/static/less/bootstrap/list-group.lessnew file+131
This diff is too big to show here.
searx/static/less/bootstrap/media.lessnew file+56
This diff is too big to show here.
searx/static/less/bootstrap/mixins.lessnew file+39
This diff is too big to show here.
searx/static/less/bootstrap/mixins/alerts.lessnew file+14
This diff is too big to show here.
searx/static/less/bootstrap/mixins/background-variant.lessnew file+8
This diff is too big to show here.
searx/static/less/bootstrap/mixins/border-radius.lessnew file+18
This diff is too big to show here.
searx/static/less/bootstrap/mixins/buttons.lessnew file+50
This diff is too big to show here.
searx/static/less/bootstrap/mixins/center-block.lessnew file+7
This diff is too big to show here.
searx/static/less/bootstrap/mixins/clearfix.lessnew file+22
This diff is too big to show here.
searx/static/less/bootstrap/mixins/forms.lessnew file+81
This diff is too big to show here.
searx/static/less/bootstrap/mixins/gradients.lessnew file+59
This diff is too big to show here.
searx/static/less/bootstrap/mixins/grid-framework.lessnew file+91
This diff is too big to show here.
searx/static/less/bootstrap/mixins/grid.lessnew file+122
This diff is too big to show here.
searx/static/less/bootstrap/mixins/hide-text.lessnew file+21
This diff is too big to show here.
searx/static/less/bootstrap/mixins/image.lessnew file+34
This diff is too big to show here.
searx/static/less/bootstrap/mixins/labels.lessnew file+12
This diff is too big to show here.
searx/static/less/bootstrap/mixins/list-group.lessnew file+29
This diff is too big to show here.
searx/static/less/bootstrap/mixins/nav-divider.lessnew file+10
This diff is too big to show here.
searx/static/less/bootstrap/mixins/nav-vertical-align.lessnew file+9
This diff is too big to show here.
searx/static/less/bootstrap/mixins/opacity.lessnew file+8
This diff is too big to show here.
searx/static/less/bootstrap/mixins/pagination.lessnew file+23
This diff is too big to show here.
searx/static/less/bootstrap/mixins/panels.lessnew file+24
This diff is too big to show here.
searx/static/less/bootstrap/mixins/progress-bar.lessnew file+10
This diff is too big to show here.
searx/static/less/bootstrap/mixins/reset-filter.lessnew file+8
This diff is too big to show here.
searx/static/less/bootstrap/mixins/resize.lessnew file+6
This diff is too big to show here.
searx/static/less/bootstrap/mixins/responsive-visibility.lessnew file+15
This diff is too big to show here.
searx/static/less/bootstrap/mixins/size.lessnew file+10
This diff is too big to show here.
searx/static/less/bootstrap/mixins/tab-focus.lessnew file+9
This diff is too big to show here.
searx/static/less/bootstrap/mixins/table-row.lessnew file+28
This diff is too big to show here.
searx/static/less/bootstrap/mixins/text-emphasis.lessnew file+8
This diff is too big to show here.
searx/static/less/bootstrap/mixins/text-overflow.lessnew file+8
This diff is too big to show here.
searx/static/less/bootstrap/mixins/vendor-prefixes.lessnew file+224
This diff is too big to show here.
searx/static/less/bootstrap/modals.lessnew file+150
This diff is too big to show here.
searx/static/less/bootstrap/navbar.lessnew file+655
This diff is too big to show here.
searx/static/less/bootstrap/navs.lessnew file+242
This diff is too big to show here.
searx/static/less/bootstrap/normalize.lessnew file+425
This diff is too big to show here.
searx/static/less/bootstrap/pager.lessnew file+55
This diff is too big to show here.
searx/static/less/bootstrap/pagination.lessnew file+88
This diff is too big to show here.
searx/static/less/bootstrap/panels.lessnew file+243
This diff is too big to show here.
searx/static/less/bootstrap/popovers.lessnew file+133
This diff is too big to show here.
searx/static/less/bootstrap/print.lessnew file+101
This diff is too big to show here.
searx/static/less/bootstrap/progress-bars.lessnew file+105
This diff is too big to show here.
searx/static/less/bootstrap/responsive-embed.lessnew file+34
This diff is too big to show here.
searx/static/less/bootstrap/responsive-utilities.lessnew file+194
This diff is too big to show here.
searx/static/less/bootstrap/scaffolding.lessnew file+150
This diff is too big to show here.
searx/static/less/bootstrap/tables.lessnew file+233
This diff is too big to show here.
searx/static/less/bootstrap/theme.lessnew file+258
This diff is too big to show here.
searx/static/less/bootstrap/thumbnails.lessnew file+36
This diff is too big to show here.
searx/static/less/bootstrap/tooltip.lessnew file+95
This diff is too big to show here.
searx/static/less/bootstrap/type.lessnew file+313
This diff is too big to show here.
searx/static/less/bootstrap/typeahead.lessnew file+153
This diff is too big to show here.
searx/static/less/bootstrap/utilities.lessnew file+57
This diff is too big to show here.
searx/static/less/bootstrap/variables.lessnew file+846
This diff is too big to show here.
searx/static/less/bootstrap/wells.lessnew file+29
This diff is too big to show here.
searx/static/themes/courgette/css/style-rtl.cssnew file+1
This diff is too big to show here.
searx/static/themes/courgette/css/style.cssnew file+1
This diff is too big to show here.
searx/static/themes/courgette/img/bg-body-index.jpgnew file
Binary file not shown.
searx/static/themes/courgette/img/favicon.pngnew file
Binary file not shown.
searx/static/themes/courgette/img/github_ribbon.pngnew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_500px.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_bing.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_dailymotion.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_deezer.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_deviantart.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_digg.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_duckduckgo.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_flickr.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_github.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_google play apps.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_google play movies.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_google play music.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_google.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_kickass.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_openstreetmap.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_searchcode code.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_searchcode doc.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_searchcode.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_soundcloud.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_stackoverflow.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_startpage.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_subtitleseeker.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_twitter.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_vimeo.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_wikipedia.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_yahoo.iconew file
Binary file not shown.
searx/static/themes/courgette/img/icons/icon_youtube.iconew file
Binary file not shown.
searx/static/themes/courgette/img/preference-icon.pngnew file
Binary file not shown.
searx/static/themes/courgette/img/search-icon.pngnew file
Binary file not shown.
searx/static/themes/courgette/img/searx-mobile.pngnew file
Binary file not shown.
searx/static/themes/courgette/img/searx.pngnew file
Binary file not shown.
searx/static/themes/courgette/img/searx_logo.svgnew file+203
This diff is too big to show here.
searx/static/themes/courgette/js/searx.jsnew file+45
This diff is too big to show here.
searx/static/themes/courgette/less/style-rtl.lessnew file+42
This diff is too big to show here.
searx/static/themes/courgette/less/style.lessnew file+695
This diff is too big to show here.
searx/static/themes/legacy/css/style-rtl.cssnew file+1
This diff is too big to show here.
searx/static/themes/legacy/css/style.cssnew file+1
This diff is too big to show here.
searx/static/themes/legacy/img/favicon.pngnew file
Binary file not shown.
searx/static/themes/legacy/img/github_ribbon.pngnew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_500px.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_bing.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_dailymotion.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_deezer.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_deviantart.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_digg.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_duckduckgo.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_flickr.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_github.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_google play apps.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_google play movies.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_google play music.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_google.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_kickass.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_openstreetmap.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_searchcode code.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_searchcode doc.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_searchcode.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_soundcloud.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_stackoverflow.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_startpage.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_subtitleseeker.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_twitter.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_vimeo.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_wikipedia.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_yahoo.iconew file
Binary file not shown.
searx/static/themes/legacy/img/icons/icon_youtube.iconew file
Binary file not shown.
searx/static/themes/legacy/img/preference-icon.pngnew file
Binary file not shown.
searx/static/themes/legacy/img/search-icon.pngnew file
Binary file not shown.
searx/static/themes/legacy/img/searx.pngnew file
Binary file not shown.
searx/static/themes/legacy/img/searx_logo.svgnew file+203
This diff is too big to show here.
searx/static/themes/legacy/js/searx.jsnew file+49
This diff is too big to show here.
searx/static/themes/legacy/less/autocompleter.lessnew file+61
This diff is too big to show here.
searx/static/themes/legacy/less/code.lessnew file+83
This diff is too big to show here.
searx/static/themes/legacy/less/definitions.lessnew file+119
This diff is too big to show here.
searx/static/themes/legacy/less/mixins.lessnew file+27
This diff is too big to show here.
searx/static/themes/legacy/less/search.lessnew file+68
This diff is too big to show here.
searx/static/themes/legacy/less/style-rtl.lessnew file+11
This diff is too big to show here.
searx/static/themes/legacy/less/style.lessnew file+747
This diff is too big to show here.
searx/static/themes/oscar/README.rst−39
This diff is too big to show here.
searx/static/themes/oscar/css/bootstrap-theme.cssdeleted−587
This diff is too big to show here.
searx/static/themes/oscar/css/bootstrap-theme.min.cssdeleted−6
This diff is too big to show here.
searx/static/themes/oscar/css/bootstrap-theme.min.css.mapdeleted−1
This diff is too big to show here.
searx/static/themes/oscar/css/bootstrap.cssdeleted−6990
This diff is too big to show here.
searx/static/themes/oscar/css/bootstrap.min.cssdeleted−5
This diff is too big to show here.
searx/static/themes/oscar/css/bootstrap.min.css.mapdeleted−1
This diff is too big to show here.
searx/static/themes/oscar/css/images/layers-2x.pngdeleted
Binary file not shown.
searx/static/themes/oscar/css/images/layers.pngdeleted
Binary file not shown.
searx/static/themes/oscar/css/images/marker-icon-2x.pngdeleted
Binary file not shown.
searx/static/themes/oscar/css/images/marker-icon.pngdeleted
Binary file not shown.
searx/static/themes/oscar/css/images/marker-shadow.pngdeleted
Binary file not shown.
searx/static/themes/oscar/css/leaflet.cssdeleted−640
This diff is too big to show here.
searx/static/themes/oscar/css/leaflet.min.css+93−1
This diff is too big to show here.
searx/static/themes/oscar/css/leaflet.min.css.mapdeleted−1
This diff is too big to show here.
searx/static/themes/oscar/css/logicodev-dark.css+175−283
This diff is too big to show here.
searx/static/themes/oscar/css/logicodev-dark.min.css+1−1
This diff is too big to show here.
searx/static/themes/oscar/css/logicodev-dark.min.css.mapdeleted−1
This diff is too big to show here.
searx/static/themes/oscar/css/logicodev.css+178−296
This diff is too big to show here.
searx/static/themes/oscar/css/logicodev.min.css+1−1
This diff is too big to show here.
searx/static/themes/oscar/css/logicodev.min.css.mapdeleted−1
This diff is too big to show here.
searx/static/themes/oscar/css/pointhi.css+104−185
This diff is too big to show here.
searx/static/themes/oscar/css/pointhi.min.css+1−1
This diff is too big to show here.
searx/static/themes/oscar/css/pointhi.min.css.mapdeleted−1
This diff is too big to show here.
searx/static/themes/oscar/fonts/glyphicons-halflings-regular.eotdeleted
Binary file not shown.
searx/static/themes/oscar/fonts/glyphicons-halflings-regular.svgdeleted−288
This diff is too big to show here.
searx/static/themes/oscar/fonts/glyphicons-halflings-regular.ttfdeleted
Binary file not shown.
searx/static/themes/oscar/fonts/glyphicons-halflings-regular.woffdeleted
Binary file not shown.
searx/static/themes/oscar/fonts/glyphicons-halflings-regular.woff2deleted
Binary file not shown.
searx/static/themes/oscar/gruntfile.js+29−87
This diff is too big to show here.
searx/static/themes/oscar/img/icons/bandcamp.pngdeleted
Binary file not shown.
searx/static/themes/oscar/img/map/layers-2x.pngnew file
Binary file not shown.
searx/static/themes/oscar/img/map/layers.pngnew file
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon-2x-green.pngnew file
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon-2x-orange.pngnew file
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon-2x-red.pngnew file
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon-2x.pngnew file
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon-green.pngnew file
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon-orange.pngnew file
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon-red.pngnew file
Binary file not shown.
searx/static/themes/oscar/img/map/marker-icon.pngnew file
Binary file not shown.
searx/static/themes/oscar/img/map/marker-shadow.pngnew file
Binary file not shown.
searx/static/themes/oscar/js/bootstrap.bundle.min.jsdeleted−7
This diff is too big to show here.
searx/static/themes/oscar/js/bootstrap.min.jsdeleted−6
This diff is too big to show here.
searx/static/themes/oscar/js/jquery.min.jsdeleted−2
This diff is too big to show here.
searx/static/themes/oscar/js/leaflet-0.7.3.min.jsnew file+9
This diff is too big to show here.
searx/static/themes/oscar/js/leaflet.jsdeleted−6
This diff is too big to show here.
searx/static/themes/oscar/js/searx.js+158−238
This diff is too big to show here.
searx/static/themes/oscar/js/searx.min.js+2−4
This diff is too big to show here.
searx/static/themes/oscar/js/searx.min.js.mapdeleted−1
This diff is too big to show here.
searx/static/themes/oscar/js/searx_src/00_requirejs_config.jsnew file+23
This diff is too big to show here.
searx/static/themes/oscar/src/js/01_init.js → searx/static/themes/oscar/js/searx_src/01_init.jsrenamed+1−5
This diff is too big to show here.
searx/static/themes/oscar/src/js/autocompleter.js → searx/static/themes/oscar/js/searx_src/autocompleter.jsrenamed+13−24
This diff is too big to show here.
searx/static/themes/oscar/src/js/element_modifiers.js → searx/static/themes/oscar/js/searx_src/element_modifiers.jsrenamed−13
This diff is too big to show here.
searx/static/themes/oscar/src/js/infobox.js → searx/static/themes/oscar/js/searx_src/infobox.jsrenamed
The content is the same.
searx/static/themes/oscar/js/searx_src/leaflet_map.jsnew file+167
This diff is too big to show here.
searx/static/themes/oscar/src/js/toggleall.js → searx/static/themes/oscar/js/searx_src/toggleall.jsrenamed
The content is the same.
searx/static/themes/oscar/js/typeahead.bundle.min.js−9
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev-dark/oscar.less+4−16
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev/advanced.less
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev/checkbox.less
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev/code.less+101
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev/cursor.less
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev/footer.less
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev/infobox.less−5
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev/navbar.less
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev/onoff.less
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev/oscar.less
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev/preferences.less+15
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev/results.less+7−81
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev/search.less+10−5
This diff is too big to show here.
searx/static/themes/oscar/less/logicodev/variables.less
This diff is too big to show here.
searx/static/themes/oscar/less/pointhi/advanced.less
This diff is too big to show here.
searx/static/themes/oscar/less/pointhi/checkbox.less
This diff is too big to show here.
searx/static/themes/oscar/less/pointhi/code.less+79
This diff is too big to show here.
searx/static/themes/oscar/less/pointhi/cursor.less
This diff is too big to show here.
searx/static/themes/oscar/less/pointhi/footer.less
This diff is too big to show here.
searx/static/themes/oscar/less/pointhi/infobox.less
This diff is too big to show here.
searx/static/themes/oscar/less/pointhi/navbar.less
This diff is too big to show here.
searx/static/themes/oscar/less/pointhi/onoff.less
This diff is too big to show here.
searx/static/themes/oscar/less/pointhi/oscar.less−2
This diff is too big to show here.
searx/static/themes/oscar/less/pointhi/preferences.less+3
This diff is too big to show here.
searx/static/themes/oscar/less/pointhi/results.less−4
This diff is too big to show here.
searx/static/themes/oscar/less/pointhi/search.less
This diff is too big to show here.
searx/static/themes/oscar/package.json+7−12
This diff is too big to show here.
searx/static/themes/oscar/src/js/leaflet_map.js−79
This diff is too big to show here.
searx/static/themes/oscar/src/less/bootstrap/bootstrap.less−3
This diff is too big to show here.
searx/static/themes/oscar/src/less/bootstrap/typeahead.less−154
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/code.less−14
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/preferences.less−31
This diff is too big to show here.
searx/static/themes/oscar/src/less/logicodev/pygments.less−99
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/code.less−2
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/preferences.less−19
This diff is too big to show here.
searx/static/themes/oscar/src/less/pointhi/pygments.less−99
This diff is too big to show here.
searx/static/themes/pix-art/css/style.css+1
This diff is too big to show here.
searx/static/themes/pix-art/img/favicon.png
This diff is too big to show here.
searx/static/themes/pix-art/img/preference-icon-pixel.png
This diff is too big to show here.
searx/static/themes/pix-art/img/search-icon-pixel.png
This diff is too big to show here.
searx/static/themes/pix-art/img/searx-pixel-small.png
This diff is too big to show here.
searx/static/themes/pix-art/img/searx-pixel.png
This diff is too big to show here.
searx/static/themes/pix-art/js/searx.js+141
This diff is too big to show here.
searx/static/themes/pix-art/less/definitions.less+119
This diff is too big to show here.
searx/static/themes/pix-art/less/mixins.less+27
This diff is too big to show here.
searx/static/themes/pix-art/less/search.less+57
This diff is too big to show here.
searx/static/themes/pix-art/less/style.less+451
This diff is too big to show here.
searx/static/themes/simple/css/searx-rtl.css+83−164
This diff is too big to show here.
searx/static/themes/simple/css/searx-rtl.min.css+1−1
This diff is too big to show here.
searx/static/themes/simple/css/searx.css+83−164
This diff is too big to show here.
searx/static/themes/simple/css/searx.min.css+1−1
This diff is too big to show here.
searx/static/themes/simple/gruntfile.js+2−2
This diff is too big to show here.
searx/static/themes/simple/js/searx.head.js+1−1
This diff is too big to show here.
searx/static/themes/simple/js/searx.head.min.js+2−2
This diff is too big to show here.
searx/static/themes/simple/js/searx.head.min.js.map+1−1
This diff is too big to show here.
searx/static/themes/simple/js/searx.js+240−165
This diff is too big to show here.
searx/static/themes/simple/js/searx.min.js+4−5
This diff is too big to show here.
searx/static/themes/simple/js/searx.min.js.map+1−1
This diff is too big to show here.
searx/static/themes/simple/js/searx_head/00_init.js+1−1
This diff is too big to show here.
searx/static/themes/simple/js/searx_src/searx_imageresult.js+49−61
This diff is too big to show here.
searx/static/themes/simple/js/searx_src/searx_mapresult.js+87
This diff is too big to show here.
searx/static/themes/simple/js/searx_src/searx_results.js+1−1
This diff is too big to show here.
searx/static/themes/simple/js/searx_src/searx_search.js+1−1
This diff is too big to show here.
searx/static/themes/simple/less/code.less+80−5
This diff is too big to show here.
searx/static/themes/simple/less/preferences.less−17
This diff is too big to show here.
searx/static/themes/simple/less/pygments.less−99
This diff is too big to show here.
searx/static/themes/simple/less/style.less−32
This diff is too big to show here.
searx/templates/__common__/opensearch.xml+2−2
This diff is too big to show here.
searx/templates/__common__/translations.js.tpl+1
This diff is too big to show here.
searx/templates/courgette/404.html+9
This diff is too big to show here.
searx/templates/courgette/about.html+5
This diff is too big to show here.
searx/templates/courgette/base.html+43
This diff is too big to show here.
searx/templates/courgette/categories.html+9
This diff is too big to show here.
searx/templates/courgette/color.css+34
This diff is too big to show here.
searx/templates/courgette/github_ribbon.html+3
This diff is too big to show here.
searx/templates/courgette/index.html+17
This diff is too big to show here.
searx/templates/courgette/preferences.html+150
This diff is too big to show here.
searx/templates/courgette/result_templates/code.html+11
This diff is too big to show here.
searx/templates/courgette/result_templates/default.html+13
This diff is too big to show here.
searx/templates/courgette/result_templates/images.html+6
This diff is too big to show here.
searx/templates/courgette/result_templates/key-value.html+13
This diff is too big to show here.
searx/templates/courgette/result_templates/map.html+13
This diff is too big to show here.
searx/templates/courgette/result_templates/torrent.html+13
This diff is too big to show here.
searx/templates/courgette/result_templates/videos.html+10
This diff is too big to show here.
searx/templates/courgette/results.html+87
This diff is too big to show here.
searx/templates/courgette/search.html+7
This diff is too big to show here.
searx/templates/courgette/stats.html+22
This diff is too big to show here.
searx/templates/legacy/404.html+9
This diff is too big to show here.
searx/templates/legacy/about.html+5
This diff is too big to show here.
searx/templates/legacy/base.html+38
This diff is too big to show here.
searx/templates/legacy/categories.html+10
This diff is too big to show here.
searx/templates/legacy/github_ribbon.html+3
This diff is too big to show here.
searx/templates/legacy/index.html+18
This diff is too big to show here.
searx/templates/legacy/infobox.html+51
This diff is too big to show here.
searx/templates/legacy/preferences.html+147
This diff is too big to show here.
searx/templates/legacy/result_templates/code.html+11
This diff is too big to show here.
searx/templates/legacy/result_templates/default.html+11
This diff is too big to show here.
searx/templates/legacy/result_templates/images.html+6
This diff is too big to show here.
searx/templates/legacy/result_templates/key-value.html+13
This diff is too big to show here.
searx/templates/legacy/result_templates/map.html+13
This diff is too big to show here.
searx/templates/legacy/result_templates/torrent.html+13
This diff is too big to show here.
searx/templates/legacy/result_templates/videos.html+6
This diff is too big to show here.
searx/templates/legacy/results.html+104
This diff is too big to show here.
searx/templates/legacy/search.html+8
This diff is too big to show here.
searx/templates/legacy/stats.html+22
This diff is too big to show here.
searx/templates/oscar/base.html+15−14
This diff is too big to show here.
searx/templates/oscar/index.html+8−7
This diff is too big to show here.
searx/templates/oscar/macros.html+44−70
This diff is too big to show here.
searx/templates/oscar/messages/no_results.html+1−1
This diff is too big to show here.
searx/templates/oscar/preferences.html+10−33
This diff is too big to show here.
searx/templates/oscar/result_templates/default.html+7−7
This diff is too big to show here.
searx/templates/oscar/result_templates/files.html−55
This diff is too big to show here.
searx/templates/oscar/result_templates/key-value.html+2−2
This diff is too big to show here.
searx/templates/oscar/result_templates/map.html+56−60
This diff is too big to show here.
searx/templates/oscar/result_templates/products.html+1−2
This diff is too big to show here.
searx/templates/oscar/result_templates/torrent.html+15−15
This diff is too big to show here.
searx/templates/oscar/result_templates/videos.html+16−16
This diff is too big to show here.
searx/templates/oscar/results.html+1−12
This diff is too big to show here.
searx/templates/oscar/search.html+1−1
This diff is too big to show here.
searx/templates/pix-art/404.html+9
This diff is too big to show here.
searx/templates/pix-art/about.html+4
This diff is too big to show here.
searx/templates/pix-art/base.html+35
This diff is too big to show here.
searx/templates/pix-art/index.html+12
This diff is too big to show here.
searx/templates/pix-art/preferences.html+90
This diff is too big to show here.
searx/templates/pix-art/result_templates/default.html+7
This diff is too big to show here.
searx/templates/pix-art/result_templates/images.html+6
This diff is too big to show here.
searx/templates/pix-art/results.html+32
This diff is too big to show here.
searx/templates/pix-art/search.html+9
This diff is too big to show here.
searx/templates/pix-art/stats.html+22
This diff is too big to show here.
searx/templates/simple/base.html+3−3
This diff is too big to show here.
searx/templates/simple/macros.html+2−2
This diff is too big to show here.
searx/templates/simple/messages/no_results.html+1−1
This diff is too big to show here.
searx/templates/simple/preferences.html+3−14
This diff is too big to show here.
searx/templates/simple/result_templates/map.html+52−35
This diff is too big to show here.
searx/templates/simple/results.html−9
This diff is too big to show here.
searx/testing.py+3−3
This diff is too big to show here.
searx/translations/ar/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/bg/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/bo/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/ca/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/cs/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/cs/LC_MESSAGES/messages.po+92−93
This diff is too big to show here.
searx/translations/cy/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/da/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/de/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/de/LC_MESSAGES/messages.po+19−21
This diff is too big to show here.
searx/translations/el_GR/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/en/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/eo/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/es/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/es/LC_MESSAGES/messages.po+19−20
This diff is too big to show here.
searx/translations/et/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/et/LC_MESSAGES/messages.po+18−18
This diff is too big to show here.
searx/translations/eu/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/eu/LC_MESSAGES/messages.po+21−22
This diff is too big to show here.
searx/translations/fa_IR/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/fi/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/fil/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/fr/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/fr/LC_MESSAGES/messages.po+21−21
This diff is too big to show here.
searx/translations/gl/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/gl/LC_MESSAGES/messages.po+51−52
This diff is too big to show here.
searx/translations/he/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/he/LC_MESSAGES/messages.po+17−18
This diff is too big to show here.
searx/translations/hr/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/hr/LC_MESSAGES/messages.po+22−23
This diff is too big to show here.
searx/translations/hu/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/ia/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/it/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/it/LC_MESSAGES/messages.po+23−23
This diff is too big to show here.
searx/translations/ja/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/lt/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/lt/LC_MESSAGES/messages.po+24−24
This diff is too big to show here.
searx/translations/nl/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/nl/LC_MESSAGES/messages.po+20−21
This diff is too big to show here.
searx/translations/nl_BE/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/nl_BE/LC_MESSAGES/messages.po+19−20
This diff is too big to show here.
searx/translations/oc/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/pl/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/pt/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/pt_BR/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/pt_BR/LC_MESSAGES/messages.po+19−20
This diff is too big to show here.
searx/translations/ro/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/ru/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/ru/LC_MESSAGES/messages.po+21−22
This diff is too big to show here.
searx/translations/sk/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/sl/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/sr/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/sr/LC_MESSAGES/messages.po+23−24
This diff is too big to show here.
searx/translations/sv/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/sv/LC_MESSAGES/messages.po+22−22
This diff is too big to show here.
searx/translations/ta/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/te/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/tr/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/tr/LC_MESSAGES/messages.po+134−135
This diff is too big to show here.
searx/translations/uk/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/vi/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/zh_CN/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/zh_CN/LC_MESSAGES/messages.po+20−21
This diff is too big to show here.
searx/translations/zh_Hant_TW/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/zh_TW/LC_MESSAGES/messages.mo
This diff is too big to show here.
searx/translations/zh_TW/LC_MESSAGES/messages.po+18−19
This diff is too big to show here.
searx/utils.py+57−147
This diff is too big to show here.
searx/version.py+2−2
This diff is too big to show here.
searx/webadapter.py+26−21
This diff is too big to show here.
searx/webapp.py+127−264
This diff is too big to show here.
searx/webutils.py+1−4
This diff is too big to show here.
searx_extra/__init__.py
This diff is too big to show here.
searx_extra/update/__init__.py
This diff is too big to show here.
searx_extra/update/update_currencies.py−150
This diff is too big to show here.
searx_extra/update/update_engine_descriptions.py−202
This diff is too big to show here.
searx_extra/update/update_external_bangs.py−157
This diff is too big to show here.
searx_extra/update/update_osm_keys_tags.py−209
This diff is too big to show here.
searx_extra/update/update_pygments.py−138
This diff is too big to show here.
searx_extra/update/update_wikidata_units.py−54
This diff is too big to show here.
setup.py+6−5
This diff is too big to show here.
tests/test_robot.py+23
This diff is too big to show here.
tests/unit/engines/test_command.py+1
This diff is too big to show here.
tests/unit/network/__init__.py
This diff is too big to show here.
tests/unit/network/test_network.py−239
This diff is too big to show here.
tests/unit/settings/empty_settings.yml
This diff is too big to show here.
tests/unit/settings/syntaxerror_settings.yml−2
This diff is too big to show here.
tests/unit/settings/user_settings.yml−111
This diff is too big to show here.
tests/unit/settings/user_settings_keep_only.yml−14
This diff is too big to show here.
tests/unit/settings/user_settings_remove.yml−10
This diff is too big to show here.
tests/unit/settings/user_settings_remove2.yml−15
This diff is too big to show here.
tests/unit/settings/user_settings_simple.yml−9
This diff is too big to show here.
tests/unit/test_engines_init.py+1−1
This diff is too big to show here.
tests/unit/test_external_bangs.py−117
This diff is too big to show here.
tests/unit/test_plugins.py+2−2
This diff is too big to show here.
tests/unit/test_preferences.py+12−8
This diff is too big to show here.
tests/unit/test_query.py−192
This diff is too big to show here.
tests/unit/test_results.py+4−4
This diff is too big to show here.
tests/unit/test_search.py+9−22
This diff is too big to show here.
tests/unit/test_settings_loader.py−122
This diff is too big to show here.
tests/unit/test_standalone_searx.py+20−13
This diff is too big to show here.
tests/unit/test_utils.py+2−92
This diff is too big to show here.
tests/unit/test_webadapter.py+3−3
This diff is too big to show here.
tests/unit/test_webapp.py+12−63
This diff is too big to show here.
tests/unit/test_webutils.py−22
This diff is too big to show here.
utils/brand.env+1−4
This diff is too big to show here.
utils/build_env.py−38
This diff is too big to show here.
utils/fabfile.py+117
This diff is too big to show here.
utils/fetch_ahmia_blacklist.py+4−1
This diff is too big to show here.
utils/fetch_currencies.py+163
This diff is too big to show here.
utils/fetch_firefox_version.py+6−1
This diff is too big to show here.
utils/fetch_languages.py+8−6
This diff is too big to show here.
utils/fetch_wikidata_units.py+47
This diff is too big to show here.
utils/filtron.sh+2−2
This diff is too big to show here.
utils/google_search.py
This diff is too big to show here.
utils/lib.sh+17−327
This diff is too big to show here.
utils/lxc-searx.env+4−3
This diff is too big to show here.
utils/lxc.sh+9−10
This diff is too big to show here.
utils/makefile.include+51−12
This diff is too big to show here.
utils/makefile.lxc+2−5
This diff is too big to show here.
utils/makefile.python+263
This diff is too big to show here.
utils/makefile.sphinx+201
This diff is too big to show here.
utils/searx.sh+17−27
This diff is too big to show here.
utils/site-python/sphinx_build_tools.py+48
This diff is too big to show here.
utils/standalone_searx.py+8−7
This diff is too big to show here.
utils/templates/etc/nginx/default.apps-available/morty.conf+1−1
This diff is too big to show here.
utils/templates/etc/nginx/default.apps-available/searx.conf:filtron+3−3
This diff is too big to show here.
utils/templates/etc/searx/use_default_settings.yml−22
This diff is too big to show here.
utils/templates/etc/uwsgi/apps-archlinux/searx.ini+1−4
This diff is too big to show here.
utils/templates/etc/uwsgi/apps-available/searx.ini−3
This diff is too big to show here.
utils/update-translations.sh
This diff is too big to show here.