1. Setting up a self hosted searx instance on a VPS

Searx-ng is a free and open-source metasearch engine, which means it aggregates search results from various search engines while respecting user privacy. SearxNG aggregates results from multiple search services, it also removes all private data from requests going to these search services. You can also create your own custom engines and plugins.

Let's setup a custom instance of searx with our own functionallity

Setting up a VPS

We need a server to host your instance of Searx-ng. This can be a physical server or a virtual private server (VPS) from a provider like AWS, DigitalOcean, or Linode. I already have the pubic IP address of my VPS to a domain that I control

$ nslookup search.eliasmanj.io
Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
Name:	search.eliasmanj.io
Address: 216.238.87.188

In my domain registrar, I have already setup the DNS host records for search.eliasmanj.io to point to the IP address of my VPS I will be hosting the searxng server

Let's install a generic instance on searxng first and check everything is working, inside our newly created VPS

Install docker and git dependencies

sudo apt install git docker.io docker-compose

Accept incoming connections to port 80

sudo udw allow http
sudo udw allow htts

Go to the searxng-docker repository and clone it to /usr/local

cd /usr/local
git clone https://github.com/searxng/searxng-docker.git
cd searxng-docker

Let's take a look at the docker-compose.yaml file of the repository, it has three services: Caddy, Redis, and Searxng.

Caddy is a popular open-source web server written in Go. It's known for its simplicity, ease of use, and automatic HTTPS configuration using Let's Encrypt by default. Caddy automatically obtains and renews TLS certificates for your sites using Let's Encrypt, ensuring secure connections by default.

Lets break down the docker-compose services and parameters

Caddy:
- It sets the container name to caddy
- Uses the host network mode for networking (network_mode: host).
- Mounts volumes for Caddyfile configuration, data, and configuration files.
- Sets environment variables for hostname and TLS configuration.
- Drops all Linux capabilities (cap_drop: ALL) except for NET_BIND_SERVICE.
Redis:
- A redis db instance using the official Redis image based on Alpine Linux
- Adds the service to the searxng"network.
- Mounts a volume for Redis data.
Searxng:
- Uses the searxng/searxng image with the latest tag and adds the service to the searxngnetwork.
- Maps port 8080 on localhost to port 8080 inside the container.
- Mounts a volume for Searxng configuration.
- Sets an environment variable for the base URL.
- Configures logging

Run searxng service

Create a secure secret key and put the key inside the searxng/settings.yml file

Example:

openssl rand -base64 36
y+hd6EfTcmHC3hh7vM++Y+ZykTBRoOQ6kJ3vOy6mqO73wWXD

There is also the .env file, here we put the instance hostname and an email for the let's encrypt certificate

# By default listen on https://localhost
# To change this:
# * uncomment SEARXNG_HOSTNAME, and replace <host> by the SearXNG hostname
# * uncomment LETSENCRYPT_EMAIL, and replace <email> by your email (require to create a Let's Encrypt certificate)

# SEARXNG_HOSTNAME=<host>
# LETSENCRYPT_EMAIL=<email>

To start the searxng instance in the background

docker-compose up -d

The generic service should now be up and running on the search.eliasmanj.io domain

curl -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9" -H "Accept-Language: en-US,en;q=0.9" -H "Connection: keep-alive" -H "Upgrade-Insecure-Requests: 1" "https://search.eliasmanj.io/search"


<!DOCTYPE html>
<html class="no-js theme-dark center-aligment-no" lang="en-EN" >
<head>
  <meta charset="UTF-8" />
  <meta name="description" content="SearXNG — a privacy-respecting, open metasearch engine">
  <meta name="keywords" content="SearXNG, search, search engine, metasearch, meta search">
...

The searxng-docker repository also has a template for systemctl service

[Unit]
Description=SearXNG service
Requires=docker.service
After=docker.service

[Service]
Restart=on-failure

Environment=SEARXNG_DOCKERCOMPOSEFILE=docker-compose.yaml

WorkingDirectory=/usr/local/searxng-docker
ExecStart=/usr/local/bin/docker-compose -f ${SEARXNG_DOCKERCOMPOSEFILE} up --remove-orphans
ExecStop=/usr/local/bin/docker-compose -f ${SEARXNG_DOCKERCOMPOSEFILE} down

[Install]
WantedBy=multi-user.target

We can modify this template to add the searxng docker instance to run at startup, in my case, the docker-compose binary is located at /usr/bin/docker-compose

After creating our custom service template we can enable the service

systemctl enable /usr/local/searxng-docker/searxng-docker.service

And start the service

systemctl start searxng-docker.service

Now that the seaxng instance is working fine, we can continue to customize the searx code

Checking image

Inside server

docker cp <container_id_or_name>:/usr/local/searxng/searx/static/themes/simple/img/searxng.png /usr/local/searxng.png

Inside local pc

scp root@216.238.87.188:/usr/local/searxng.png /home/emanjarrez/Pictures/searxng.png

Copy to server

scp /home/emanjarrez/code/searxng-nomedia/searx/static/themes/simple/img/searxng.png root@216.238.87.188:/usr/local/searxng.png

Copy to docker

docker cp /usr/local/searxng.png  <container_id_or_name>:/usr/local/searxng/searx/static/themes/simple/img/searxng.png

Next: 2. Creating a custom searxng plugin in Python