Most search engines now bolt an AI answer box onto the top of the results page. That can be useful, but it also means your query and whatever the model does with it are happening on somebody else\u2019s infrastructure.<\/p>\n\n\n\n
This project builds the same basic workflow locally:<\/p>\n\n\n\n
\/search\/<\/code> on an existing site.<\/li>\n<\/ul>\n\n\n\nWhat the final setup looks like<\/h2>\n\n\n\n
The public page is:<\/p>\n\n\n\n
https://YOUR_DOMAIN\/search\/<\/code><\/pre>\n\n\n\nThe local services are:<\/p>\n\n\n\n
SearXNG: http:\/\/127.0.0.1:8080\nOllama: http:\/\/127.0.0.1:11434\nAI search: http:\/\/0.0.0.0:5001<\/code><\/pre>\n\n\n\nOf course, you can also use subdomains instead of directory based; I started using directories ages ago and have too much momentum to care about changing now.<\/p>\n\n\n\n
Not every search needs a summary. Sometimes you just want results. The browser hits the AI search wrapper which loads normal SearXNG results and only calls Ollama when the user requests it.<\/p>\n\n\n\n
Assumptions<\/h2>\n\n\n\n
This guide is based on my experience setting this up on my own rig. It should apply broadly to current Ubuntu and derivatives, possibly with some tinkering:<\/p>\n\n\n\n
\n- Ubuntu 25.10 or close enough.<\/li>\n\n\n\n
- Docker Engine and Compose v2.<\/li>\n\n\n\n
- Apache 2.4 as the reverse proxy (used in this doc, can be easily adapted for other RPs)<\/li>\n\n\n\n
- A machine that can run Ollama locally (my machine for reference: Ryzen9 3900X, 128GiB RAM, NVIDIA 4060Ti 16GiB VRAM).<\/li>\n\n\n\n
- A reverse proxy path of
\/search\/<\/code>.<\/li>\n<\/ul>\n\n\n\nThe commands use \/opt\/ai-search<\/code>. Change that path if you want, but don\u2019t scatter the files around. Future-you will hate present-you.<\/p>\n\n\n\nInstall Docker from the Docker repository<\/h2>\n\n\n\n
Ubuntu\u2019s Docker packages and Docker\u2019s official packages can conflict. Pick one lane. I use Docker\u2019s repository here.<\/p>\n\n\n\n
sudo apt update\nsudo apt install -y ca-certificates curl gnupg apache2\n\nsudo install -m 0755 -d \/etc\/apt\/keyrings\nsudo curl -fsSL https:\/\/download.docker.com\/linux\/ubuntu\/gpg \\\n -o \/etc\/apt\/keyrings\/docker.asc\nsudo chmod a+r \/etc\/apt\/keyrings\/docker.asc\n\necho \\\n\"deb [arch=$(dpkg --print-architecture) signed-by=\/etc\/apt\/keyrings\/docker.asc] \\\nhttps:\/\/download.docker.com\/linux\/ubuntu \\\n$(. \/etc\/os-release && echo \"$VERSION_CODENAME\") stable\" | \\\nsudo tee \/etc\/apt\/sources.list.d\/docker.list > \/dev\/null\n\nsudo apt update\nsudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin\nsudo systemctl enable --now docker<\/code><\/pre>\n\n\n\nCheck it:<\/p>\n\n\n\n
sudo docker version\ndocker compose version<\/code><\/pre>\n\n\n\nCreate the project directory<\/h2>\n\n\n\nsudo mkdir -p \/opt\/ai-search\/{searxng,ollama}\nsudo chown -R \"$USER:$USER\" \/opt\/ai-search\ncd \/opt\/ai-search<\/code><\/pre>\n\n\n\nDocker Compose: SearXNG and Ollama<\/h2>\n\n\n\n
This setup uses Docker host networking. That is deliberate.<\/p>\n\n\n\n
I use a userspace VPN and systemwide TailScale on the same rig (getting my money’s worth out of a gaming machine when I’m not gaming). Docker bridge networking and Docker\u2019s embedded DNS can get weird and create frustrating, time wasting conflicts. Host networking removes that whole layer for this project. The tradeoff is that ports bind directly on the host, so do not run this blindly on a shared machine.<\/p>\n\n\n\n
Create \/opt\/ai-search\/docker-compose.yml<\/code>:<\/p>\n\n\n\nservices:\n searxng:\n image: docker.io\/searxng\/searxng:latest\n container_name: searxng\n restart: unless-stopped\n network_mode: host\n volumes:\n - .\/searxng:\/etc\/searxng\n\n ollama:\n image: docker.io\/ollama\/ollama:latest\n container_name: ollama\n restart: unless-stopped\n network_mode: host\n volumes:\n - .\/ollama:\/root\/.ollama<\/code><\/pre>\n\n\n\nStart it:<\/p>\n\n\n\n
cd \/opt\/ai-search\nsudo docker compose up -d<\/code><\/pre>\n\n\n\nSearXNG settings<\/h2>\n\n\n\n
Create \/opt\/ai-search\/searxng\/settings.yml<\/code>:<\/p>\n\n\n\nuse_default_settings: true\n\ngeneral:\n debug: false\n instance_name: \"Search\"\n\nsearch:\n safe_search: 0\n autocomplete: duckduckgo\n default_lang: en-US\n formats:\n - html\n - json\n\nserver:\n secret_key: \"CHANGE_THIS_TO_A_LONG_RANDOM_VALUE\"\n base_url: http:\/\/127.0.0.1:8080\/\n limiter: false\n image_proxy: false\n method: GET\n\nui:\n infinite_scroll: false\n query_in_title: true\n results_on_new_tab: true\n\nplugins:\n searx.plugins.hostnames.SXNGPlugin:\n active: true\n searx.plugins.tracker_url_remover.SXNGPlugin:\n active: true\n searx.plugins.calculator.SXNGPlugin:\n active: true\n\nengines:\n - name: brave\n disabled: true\n - name: karmasearch\n disabled: true\n - name: karmasearch videos\n disabled: true\n - name: mojeek\n disabled: true\n - name: yahoo\n disabled: true<\/code><\/pre>\n\n\n\nGenerate a real secret key:<\/p>\n\n\n\n
python3 - <<'PY'\nimport secrets\nprint(secrets.token_hex(32))\nPY<\/code><\/pre>\n\n\n\nReplace CHANGE_THIS_TO_A_LONG_RANDOM_VALUE<\/code> with the generated value.<\/p>\n\n\n\nRestart SearXNG:<\/p>\n\n\n\n
cd \/opt\/ai-search\nsudo docker compose restart searxng\ncurl -I http:\/\/127.0.0.1:8080<\/code><\/pre>\n\n\n\nSearXNG must have JSON enabled because the AI wrapper reads search results through \/search?q=...&format=json<\/code>.<\/p>\n\n\n\nPull a local model with Ollama<\/h2>\n\n\n\n
This uses Qwen 2.5 7B because it is small enough for normal local hardware and good enough for short search summaries.
f you use a<\/p>\n\n\n\n
curl http:\/\/127.0.0.1:11434\/api\/pull \\\n -d '{\"model\":\"qwen2.5:7b\",\"stream\":false}'\n\ncurl http:\/\/127.0.0.1:11434\/api\/tags<\/code><\/pre>\n\n\n\nTest generation:<\/p>\n\n\n\n
curl http:\/\/127.0.0.1:11434\/api\/generate \\\n -d '{\"model\":\"qwen2.5:7b\",\"prompt\":\"Reply with exactly: model working\",\"stream\":false}'<\/code><\/pre>\n\n\n\nInstall Python dependencies<\/h2>\n\n\n\n
The wrapper is a small Flask app. For the service, use Gunicorn instead of Flask\u2019s development server.<\/p>\n\n\n\n
sudo apt update\nsudo apt install -y python3-flask python3-requests gunicorn<\/code><\/pre>\n\n\n\nThe AI search wrapper<\/h2>\n\n\n\n
Create \/opt\/ai-search\/ai_search_app.py<\/code>:<\/p>\n\n\n\nfrom flask import Flask, request\nimport html\nimport requests\n\napp = Flask(__name__)\n\nSEARX_UI = \"http:\/\/127.0.0.1:8080\"\nSEARX_API = \"http:\/\/127.0.0.1:8080\/search\"\nOLLAMA_API = \"http:\/\/127.0.0.1:11434\/api\/generate\"\nMODEL = \"qwen2.5:7b\"\n\nSTYLE = \"\"\"\nbody {\n margin: 0;\n font-family: system-ui, sans-serif;\n background: #111;\n color: #eee;\n}\n.topbar {\n padding: 12px;\n background: #181818;\n border-bottom: 1px solid #333;\n}\nform {\n display: flex;\n gap: 8px;\n}\ninput[type=\"text\"] {\n flex: 1;\n padding: 10px;\n font-size: 16px;\n}\nbutton {\n padding: 10px 16px;\n font-size: 16px;\n}\n.ai-box {\n padding: 14px;\n margin: 12px;\n border: 1px solid #444;\n border-radius: 8px;\n background: #1b1b1b;\n}\n.ai-loading {\n opacity: 0.75;\n}\n#raw-results {\n background: #fff;\n color: #111;\n padding: 12px;\n}\n#raw-results a {\n color: #0645ad;\n}\n\"\"\"\n\nSEARX_CSS = '<link rel=\"stylesheet\" href=\"\/search\/raw\/static\/themes\/simple\/sxng-ltr.min.css\" type=\"text\/css\">'\n\n@app.route(\"\/\")\ndef index():\n q = request.args.get(\"q\", \"\").strip()\n ai_enabled = request.args.get(\"ai\") == \"1\"\n q_html = html.escape(q)\n checked = \"checked\" if ai_enabled else \"\"\n\n if not q:\n return f\"\"\"\n<html>\n<head><title>AI Search<\/title>{SEARX_CSS}<style>{STYLE}<\/style><\/head>\n<body>\n <div class=\"topbar\">\n <form action=\"\/search\/\" method=\"get\">\n <input type=\"text\" name=\"q\" autofocus placeholder=\"Search...\" \/>\n <label style=\"display:flex;align-items:center;gap:6px\">\n <input type=\"checkbox\" name=\"ai\" value=\"1\">\n include AI\n <\/label>\n <button type=\"submit\">Search<\/button>\n <\/form>\n <\/div>\n<\/body>\n<\/html>\n\"\"\"\n\n quoted_q = requests.utils.quote(q)\n raw_url = \"\/search\/raw\/search?q=\" + quoted_q\n\n if ai_enabled:\n ai_block = \"\"\"\n <div id=\"ai\" class=\"ai-box ai-loading\">\n <b>AI Answer<\/b><br><br>\n Working...\n <\/div>\n\"\"\"\n ai_script = f\"\"\"\n <script>\n fetch(\"\/search\/answer?q=\" + encodeURIComponent({q!r}))\n .then(r => r.text())\n .then(t => {{\n document.getElementById(\"ai\").classList.remove(\"ai-loading\");\n document.getElementById(\"ai\").innerHTML = t;\n }})\n .catch(() => {{\n document.getElementById(\"ai\").innerHTML = \"<b>AI Answer<\/b><br><br>Unavailable.\";\n }});\n <\/script>\n\"\"\"\n else:\n ai_block = f\"\"\"\n <div class=\"ai-box\">\n <b>AI Answer<\/b><br><br>\n <a style=\"color:#9cf\" href=\"\/search\/?q={quoted_q}&ai=1\">Generate AI summary<\/a>\n <\/div>\n\"\"\"\n ai_script = \"\"\n\n return f\"\"\"\n<html>\n<head><title>{q_html} - AI Search<\/title>{SEARX_CSS}<style>{STYLE}<\/style><\/head>\n<body>\n <div class=\"topbar\">\n <form action=\"\/search\/\" method=\"get\">\n <input type=\"text\" name=\"q\" value=\"{q_html}\" \/>\n <label style=\"display:flex;align-items:center;gap:6px\">\n <input type=\"checkbox\" name=\"ai\" value=\"1\" {checked}>\n include AI\n <\/label>\n <button type=\"submit\">Search<\/button>\n <a style=\"color:#9cf;padding:10px\" href=\"{raw_url}\" target=\"_blank\">Open raw SearXNG<\/a>\n <\/form>\n <\/div>\n\n{ai_block}\n\n <div id=\"raw-results\">Loading search results...<\/div>\n\n <script>\n fetch(\"\/search\/raw-html?q=\" + encodeURIComponent({q!r}))\n .then(r => r.text())\n .then(t => {{\n document.getElementById(\"raw-results\").innerHTML = t;\n }})\n .catch(() => {{\n document.getElementById(\"raw-results\").innerHTML = \"Search results unavailable.\";\n }});\n <\/script>\n\n{ai_script}\n<\/body>\n<\/html>\n\"\"\"\n\n@app.route(\"\/raw-html\")\ndef raw_html():\n q = request.args.get(\"q\", \"\").strip()\n if not q:\n return \"\"\n\n r = requests.get(SEARX_UI + \"\/search\", params={\"q\": q}, timeout=45)\n r.raise_for_status()\n page = r.text\n\n start = page.find('<main id=\"main_results\"')\n if start == -1:\n return page\n\n end = page.rfind(\"<\/main>\")\n if end == -1:\n return page[start:]\n\n return page[start:end + len(\"<\/main>\")]\n\n@app.route(\"\/answer\")\ndef answer():\n q = request.args.get(\"q\", \"\").strip()\n if not q:\n return \"\"\n\n try:\n sx = requests.get(SEARX_API, params={\"q\": q, \"format\": \"json\"}, timeout=30)\n sx.raise_for_status()\n data = sx.json()\n\n lines = []\n for r in data.get(\"results\", [])[:6]:\n title = r.get(\"title\", \"\")\n content = r.get(\"content\", \"\")\n url = r.get(\"url\", \"\")\n if title or content:\n lines.append(f\"{title}\\n{content}\\n{url}\")\n\n prompt = (\n \"User query:\\n\" + q +\n \"\\n\\nSearch results:\\n\" + \"\\n\\n\".join(lines) +\n \"\\n\\nWrite a concise answer in 3 bullet points. \"\n \"Use only the provided search results. \"\n \"If the results are weak or unrelated, say so.\"\n )\n\n ol = requests.post(\n OLLAMA_API,\n json={\"model\": MODEL, \"prompt\": prompt, \"stream\": False},\n timeout=120,\n )\n ol.raise_for_status()\n\n text = ol.json().get(\"response\", \"\").strip()\n safe = html.escape(text).replace(\"\\n\", \"<br>\")\n return \"<b>AI Answer<\/b><br><br>\" + safe\n\n except Exception as e:\n return \"<b>AI Answer<\/b><br><br>Unavailable: \" + html.escape(str(e))\n\nif __name__ == \"__main__\":\n app.run(host=\"0.0.0.0\", port=5001)<\/code><\/pre>\n\n\n\nTwo small design choices are doing a lot of work here:<\/p>\n\n\n\n
\n- The search results load first.<\/li>\n\n\n\n
- AI only runs when
ai=1<\/code> is present.<\/li>\n<\/ul>\n\n\n\nThat keeps normal searches quick.<\/p>\n\n\n\n
Run it as a service<\/h2>\n\n\n\n
Create \/etc\/systemd\/system\/ai-search.service<\/code>:<\/p>\n\n\n\n[Unit]\nDescription=Local AI Search Wrapper\nAfter=network.target docker.service\nWants=docker.service\n\n[Service]\nType=simple\nUser=YOUR_LOCAL_USER\nWorkingDirectory=\/opt\/ai-search\nExecStart=\/usr\/bin\/gunicorn -w 2 -b 0.0.0.0:5001 ai_search_app:app\nRestart=always\nRestartSec=3\n\n[Install]\nWantedBy=multi-user.target<\/code><\/pre>\n\n\n\nSet your local username:<\/p>\n\n\n\n
sudo sed -i \"s\/User=YOUR_LOCAL_USER\/User=$USER\/\" \/etc\/systemd\/system\/ai-search.service\nsudo systemctl daemon-reload\nsudo systemctl enable --now ai-search.service<\/code><\/pre>\n\n\n\nTest locally:<\/p>\n\n\n\n
curl -I http:\/\/127.0.0.1:5001\ncurl -L \"http:\/\/127.0.0.1:5001\/?q=linux%20firewall\" | head\ncurl -L \"http:\/\/127.0.0.1:5001\/?q=linux%20firewall&ai=1\" | head<\/code><\/pre>\n\n\n\nApache reverse proxy<\/h2>\n\n\n\n
This example publishes the wrapper at \/search\/<\/code> and raw SearXNG at \/search\/raw\/<\/code>.<\/p>\n\n\n\nSet the backend IP before using the config:<\/p>\n\n\n\n
AI_SEARCH_HOST=\"192.168.1.50\"<\/code><\/pre>\n\n\n\nUse the actual LAN or VPN IP of the machine running the AI search service.<\/p>\n\n\n\n
Put this inside your existing Apache TLS vhost:<\/p>\n\n\n\n
# Local AI Search\n# Public paths:\n# \/search\/ -> AI wrapper\n# \/search\/raw\/ -> raw SearXNG assets and search page\n\nRedirectMatch 308 ^\/search$ \/search\/\n\nProxyPreserveHost On\nProxyRequests Off\n\n# RAW SEARXNG MUST COME FIRST\nProxyPass \/search\/raw\/ http:\/\/AI_SEARCH_HOST:8080\/\nProxyPassReverse \/search\/raw\/ http:\/\/AI_SEARCH_HOST:8080\/\n\n# AI WRAPPER SECOND\nProxyPass \/search\/ http:\/\/AI_SEARCH_HOST:5001\/\nProxyPassReverse \/search\/ http:\/\/AI_SEARCH_HOST:5001\/\n\n<Location \/search\/raw\/>\n Require all granted\n\n RequestHeader set X-Forwarded-Proto \"https\"\n RequestHeader set X-Forwarded-Host \"YOUR_DOMAIN\"\n RequestHeader set X-Forwarded-Prefix \"\/search\/raw\"\n RequestHeader set X-Scheme \"https\"\n RequestHeader set X-Script-Name \"\/search\/raw\"\n\n RequestHeader set X-Real-IP %{REMOTE_ADDR}s\n RequestHeader append X-Forwarded-For %{REMOTE_ADDR}s\n<\/Location>\n\n<Location \/search\/>\n Require all granted\n\n RequestHeader set X-Forwarded-Proto \"https\"\n RequestHeader set X-Forwarded-Host \"YOUR_DOMAIN\"\n RequestHeader set X-Forwarded-Prefix \"\/search\"\n RequestHeader set X-Scheme \"https\"\n RequestHeader set X-Script-Name \"\/search\"\n\n RequestHeader set X-Real-IP %{REMOTE_ADDR}s\n RequestHeader append X-Forwarded-For %{REMOTE_ADDR}s\n\n SetEnvIf Request_URI \"^\/search\/\" dontlog\n<\/Location><\/code><\/pre>\n\n\n\nReplace AI_SEARCH_HOST<\/code> and YOUR_DOMAIN<\/code> before reloading Apache.<\/p>\n\n\n\nThe order matters. \/search\/raw\/<\/code> must come before \/search\/<\/code>, or Apache will send raw SearXNG requests to the wrapper.<\/p>\n\n\n\nEnable modules and reload:<\/p>\n\n\n\n
sudo a2enmod proxy proxy_http headers rewrite ssl\nsudo apache2ctl configtest\nsudo systemctl reload apache2<\/code><\/pre>\n\n\n\nTest the final page<\/h2>\n\n\n\n
Open:<\/p>\n\n\n\n
https://YOUR_DOMAIN\/search\/<\/code><\/pre>\n\n\n\nSearch normally. Results should load without calling the model.<\/p>\n\n\n\n
Then check include AI<\/code> and search again. Results should still load first, and the AI answer should appear after a few seconds.<\/p>\n\n\n\nNotes from the build<\/h2>\n\n\n\n
Do not start by hacking SearXNG plugins. That sounds cleaner than it is. The current SearXNG plugin system expects proper importable Python modules and fully qualified plugin class names. A wrapper avoids tying your project to SearXNG internals.<\/p>\n\n\n\n
Do not put 127.0.0.1<\/code> URLs in HTML that will be loaded by another machine. The user\u2019s browser interprets 127.0.0.1<\/code> as the user\u2019s own computer, not your server. Use public paths like \/search\/raw\/search?...<\/code> and let Apache proxy them.<\/p>\n\n\n\nDo not run AI on every query unless you really want the latency (or you’re keeping warm in winter).<\/p>\n\n\n\n
If you use a VPN and Docker networking explodes, try host networking for this stack. It is blunt, but it avoids a lot of route and DNS drama.<\/p>\n\n\n\n
Useful references<\/h2>\n\n\n\n\n- SearXNG Search API: https:\/\/docs.searxng.org\/dev\/search_api.html<\/li>\n\n\n\n
- SearXNG JSON formats need to be enabled in
settings.yml<\/code>: https:\/\/docs.searxng.org\/dev\/search_api.html<\/li>\n\n\n\n- Ollama generate API: https:\/\/ollama.readthedocs.io\/en\/api\/<\/li>\n\n\n\n
- Ollama pull API: https:\/\/docs.ollama.com\/api\/pull<\/li>\n\n\n\n
- Docker host networking: https:\/\/docs.docker.com\/engine\/network\/drivers\/host\/<\/li>\n\n\n\n
- Apache reverse proxy guide: https:\/\/httpd.apache.org\/docs\/2.4\/howto\/reverse_proxy.html<\/li>\n\n\n\n
- Apache mod_proxy docs: https:\/\/httpd.apache.org\/docs\/current\/mod\/mod_proxy.html<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"
Most search engines now bolt an AI answer box onto the top of the results page. That can be useful, but it also means your query and whatever the model does with it are happening on somebody else\u2019s infrastructure. This project builds the same basic workflow locally: What the final setup looks like The public […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[7,3,10,9,8],"class_list":["post-26","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-ai","tag-it","tag-privacy","tag-search","tag-selfhosted"],"_links":{"self":[{"href":"https:\/\/ko4bep.net\/blog\/index.php\/wp-json\/wp\/v2\/posts\/26","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ko4bep.net\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ko4bep.net\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ko4bep.net\/blog\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ko4bep.net\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=26"}],"version-history":[{"count":1,"href":"https:\/\/ko4bep.net\/blog\/index.php\/wp-json\/wp\/v2\/posts\/26\/revisions"}],"predecessor-version":[{"id":27,"href":"https:\/\/ko4bep.net\/blog\/index.php\/wp-json\/wp\/v2\/posts\/26\/revisions\/27"}],"wp:attachment":[{"href":"https:\/\/ko4bep.net\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=26"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ko4bep.net\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=26"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ko4bep.net\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=26"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}