perf: optimise import batching

perf: use indexer's batch size for storage export

test: fix importer low-memory benchmark

chore: release v0.2.4 v0.2.4

perf: tune index backend

perf: tune storage backend

feat(storage): respect --replace flag

fix: not re-indexing despite --reindex argument

feat: improve error logging during import

test: deduplicate search test setup

chore: enable running subset of tests

test: skip if index is missing

chore: skip flake checks temporarily

chore: release v0.2.3 v0.2.3

feat: re- ingest/index on SIGUSR1/2

fix: always re-index if storage is (re-)created

feat: listen for SIGHUP if running in a terminal

chore: add check-config task

fix: invalid config generated by nixos module

fix: unable to set log level from nixos module

fix: add missing prefix to version link in footer

chore: release v0.2.2 v0.2.2

fix: error setting up programs DB

feat: disable storm internal logging

perf: use buffered channels for importing

refactor: reduce log level for storage importer

Revert "chore: fix ci failing flake check (missing dependencies)"

chore: manage simple.css with nix flake commands

chore: update bleve CLI package

chore: fix ci failing flake check (missing dependencies)

chore: add ci task

chore: fix flake check errors

feat: make web search timeout configurable

chore: simplify dev setup

test: fix missing go via golangci-lint in flake check

test: fix version flake check

fix: enable graceful shutdown

chore: reduce number of golang-ci linters

chore: release v0.2.1 v0.2.1

feat: remove entities not present in current import

fix: storage error

refactor: extract iteration from WriteIndex.Import

feat: pretty-print schema metadata file

fix: error when exporting from storage to index

feat: store import timestamp on entities

chore: release v0.2.0 v0.2.0

feat: add check-config command

refactor: remove unused http ReadCloser adapter

refactor: re-arrange importer package

style: re-organise all imports with goimports

fix: update search index as part of update timer

refactor: deduplicate indexing code

refactor: use clearer name for fetcher function

refactor(importer): use interfaces, not generics

ci: remove duplicate staticcheck hook (use golangci-lint)

test: simplify importer memory benchmarking setup

refactor!: split main function into commands

ci: specify that local imports are to be split from third-party

refactor: switch flag-parsing library

build: update flake inputs

ci: move code formatting responsibility to golangci-lint

feat: split storage import and indexing

refactor: use correct form of nolint directive

feat: store data separate from search index

refactor: pass packages by value (as options do already)

refactor: make importer.process a method

feat: add offline flag to reindex (mostly for dev)

refactor: reduce logging in import process

chore: add fetch command to justfile

refactor: rename option fields to avoid sql database quoting

refactor: use options struct for all arguments to index constructor

fix: styles broken after updates due to stale cache/hash mismatch

refactor: address linter issues

fix: handle missing manpage URL map without aborting

chore: remove nixpkgs-lint from dev shell

style: reformat yaml to 2 spaces

refactor: remove sentry monitor from import job

fix: set date of files copied from nix store

feat: improve tracking and display of import runs

feat: add --fetch action to populate data/sources

feat: make downloaded filenames consistent

fix: don't try to parse an empty last-modified header

feat: store fetched files in data directory

fix: error saving metadata when re-indexing

refactor: don't force os.Root paths to be absolute

feat: limit file operations using os.Root

refactor: instantiate AssetCollection explicitly

refactor: pass options struct to fetcher.New

refactor: store and use options struct in importer directly

refactor: split subcommand output lines more efficiently

refactor: shut down more quickly

fix: don't log and exit on graceful shutdown timeout

chore: update simple.css

chore: update go version

fix: import package even with incorrect 'homePage' casing

fix: re-enable margin-top for headers in full page view

docs: fix link to defaults.toml

feat: switch from errors to fault

chore: release v0.1.46 v0.1.46

feat: enable advanced query string syntax

feat: improve matching of multiple keywords

build: add test target to justfile

chore: release v0.1.45 v0.1.45

fix: set default values for custom sources

build: export an overlay

chore: set up treefmt-nix and reformat

docs: add status badge to readme

feat: add link to status page

fix: avoid panic when rendering combined data

fix: panic rendering corrupted data

chore: move package domain to alin.ovh

chore: release v0.1.44 v0.1.44

fix: descriptions truncated by one character

chore: update dependencies

fix: remove trailing full-stop from single-sentence descriptions

chore: release v0.1.43 v0.1.43

fix: first sentence of some descriptions cut off

build: remove call to darwin version of callPackage

chore: release v0.1.42 v0.1.42

feat: link man pages like how NixOS does it

feat: parse {man} in markdown documentation

feat: parse {file} in markdown documentation

feat: parse {var} in markdown documentation

feat: parse {env} in markdown documentation

feat: parse {command} in markdown documentation

feat: parse {option} in markdown documentation

feat: enable parsing of markdown fences

feat: create option link parser

refactor: move repository file URL generation to Repository method

refactor: make createSourceImporter a method

refactor: keep cleanup code together

refactor: remove unused metadata entry

refactor: extract http fetching to submodule

refactor: extract nixpkgs docs code to own module

build: ignore dot import linter warnings from staticcheck

chore: release v0.1.41 v0.1.41

feat: improve logging of import process

refactor: use a single field for ngram-analysed fields

feat: enable automatic re-indexing on schema version changes

fix: matching long prefixes in option names (regression)

fix: don't quit after generating index for first time

fix: keep names_with_underscores as single tokens

chore: release v0.1.40 v0.1.40

fix: exact/prefix matches for package name/attr not shown

chore: release v0.1.39 v0.1.39

chore: update flake inputs

fix: match option name prefixes even when len > 25

style: address linter complaints

chore: migrate golangci-lint configuration to v2

chore: update flake inputs

docs: update instance link in readme

chore: switch to codeberg for issue/source links

chore: release v0.1.38 v0.1.38

feat: improve display on smaller screens

fix: scroll header nav instead of wrapping words on tablets

feat: hide description table on smaller screens

chore: release v0.1.37 v0.1.37

fix: package definition link regressed since v0.1.31

fix: missing base.css in package lib folder

chore: release v0.1.36 v0.1.36

feat: prioritise whole-phrase matches

fix: prioritise exact attribute matches above all

fix: erroneously running importer on every startup

build: fix typo in .gitignore

feat: add flag to generate error page for web server

chore: release v0.1.35 v0.1.35

feat: enable searching homepages field

refactor(js): cleanup unused code

chore: remove inverted patterns from gitignore

build: remove editor-specific file (should be globally ignored)

feat: improve logging of next import time

feat: make index batch size configurable

fix: importer timeout ignored in scheduler

feat: list package platforms in detail view

chore: release v0.1.34 v0.1.34

feat: buffer fetched HTTP bodies

fix: another typo in search markup

fix: typo in indexing status

feat: re-index on startup if last run > 24 hours ago

refactor: pass indexer to importer only once in options

fix: nil pointer panic in importer

chore: release v0.1.33 v0.1.33

build: make release target abort on failure

build: simplify gitignore

fix: build failure due to outdated file reference

chore: release v0.1.32 v0.1.32

build: streamline release process

build: copy base CSS instead of linking

build: ensure justfile is not ignored

refactor: remove duplicate update flag

refactor: split out importer code from searchix/web

refactor: move SetupIndex and indexing progress to importer

fix: missing content of last index run time

feat: improve handling of long option descriptions

feat: widen body

fix: block single-character queries

feat: improve display of indexing times and durations

feat: link directly to version changelog in footer

build: show version in development

feat(js): allow opening details in new tab/window

docs: add JSDoc comments to javascript functions

docs: change status to beta

feat: promote prefix/exact name/attr matches

fix: don't attempt to load more than configured number of results

chore: release 0.1.31 v0.1.31

fix: remove rendering from search timeout restriction

perf: iterate over results without keeping the set in memory

feat(nixos-module): allow setting environment variables

feat(nixos-module): allow setting environment variables

fix: package programs displayed off-centre

feat: demote NUR results in combined search

feat: promote results with literal or prefix name matches

feat: make list of source links dynamic

feat: enable NUR package import

feat: wrap search form in semantic <search> element

build: prefix branded version with 'v'

feat: show last/next/current indexing run time

feat: shorten shutdown timeout in development

fix: footer spacing

fix: detach version from rest of footer text

docs: add build status badge

fix: wrong pagination links for combined results

feat: Convert templ components to gomponents

chore: release 0.1.30 v0.1.30

feat: show version number and link in footer

build: update base css

feat: split compound words in names into n-grams

chore: use nixpkgs-unstable as flake input

chore: release 0.1.29 v0.1.29

fix: nil pointer deference in nixpkgs-channel fetcher

chore: don't reindex on every dev start

fix: build error (incomplete commit)

chore: add build task to justfile

fix: build error caused by incomplete commit

refactor: ensure errors have stack traces

chore: release 0.1.28 v0.1.28

fix: js error due to missing class on nav menu item

build: remove lorri from direnv configuration

chore: fix nixos module default config link not updating on release

chore: release 0.1.27 v0.1.27

docs: use link to current version's defaults.toml

fix: sources disabled by default in NixOS module

chore: release 0.1.26 v0.1.26

fix: nixos import fails with default settings

chore: release 0.1.25 v0.1.25

fix: panic when source order > number of sources

chore: update flake inputs, go and templ

feat: link to referenced projects

build: add justfile alias for projectdo

chore: release 0.1.24 v0.1.24

refactor: remove unnecessary source lookup

fix(js): switching source after reset uses old query

chore: release 0.1.23 v0.1.23

fix: remove duplicate link to source code

chore: release 0.1.22 v0.1.22

feat: default to combined search on root path

refactor: interface{}->any

chore: release 0.1.21 v0.1.21

feat: prioritise exact matches on programs/attr/name field

feat: improve relevance with compound words

refactor: remove unused analyser

build(dev): automatically re-index on indexer.go changes

chore: update bleve overlay

refactor: use pointer to sourceMeta

refactor: split importer args

fix: re-index refusing to delete data directory with programs.db

chore: release 0.1.20 v0.1.20

fix: build failure (bad rebase)

chore: release 0.1.19 v0.1.19

fix: import slowdown when querying programs database

perf: enable support for cpu profiling

build: allow passing flags to just {reindex,update}

perf: pass context to importer for better tracing

chore: release 0.1.18 v0.1.18

feat: allow HTTP request logging to be disabled

feat(sentry): trace all transactions

chore: release 0.1.17 v0.1.17

build: don't try to run non-existent tests

fix(sentry): report correct HTTP path in traces

refactor: trace 10% of requests

fix: use importer configured timeout for sentry check-ins

chore: release 0.1.16 v0.1.16

feat: enable searching via program names for multi-program packages

fix(dev): fix CSP not working in dev mode behind reverse proxy

chore: release 0.1.15 v0.1.15

feat: offer to show all results on one page (if < 10000)

fix(dev): overwrite previous asset hash on change

refactor: extract pagination into module

chore: release 0.1.14 v0.1.14

fix: boost non-stemmed matches over stemmed matches

feat: show scores in results table in dev mode

chore: release 0.1.13 v0.1.13

feat: increase results per page

feat: de-rank packages on 'all' search

refactor: improve type hinting of setField

build: workaround nix build error with go 1.23

chore: update dependencies

chore: release 0.1.12 v0.1.12

fix: use UTC time for fetch/import timer

fix: enable using module in pure evaluation mode

fix: enable using module in pure evaluation mode

chore: release 0.1.11 v0.1.11

feat: apply stemming to search index terms

chore: disable integer overflow warnings

docs: update default nix-darwin importer config

chore: release 0.1.10 v0.1.10

fix: search all missing expected results

chore: release 0.1.9 v0.1.9

fix: don't error when item not found

build: remove shell.nix to force lorri to use flakes

fix: tell browsers not to capitalise search input

chore: release 0.1.8 v0.1.8

build: update nix package version number automatically

fix: nil panics rendering last page of search results

fix: panic rendering error pages

refactor: ignore unused variable

chore: release 0.1.8

fix: opensearch searches not using input query

refactor: remove unnecessary DocumentData struct

feat: enable combined search

fix: extra data after error parsing query string

fix: panic rendering search results

feat: more structured logging

fix: panic on search error

chore: release 0.1.7 v0.1.7

build: add command to cut a release

build: generate correct changelog for next version

build: stamp binary correctly

refactor: rename go module

build: remove formatter for go template files

build: remove formatter for go template files

feat: add message for users with javascript disabled

fix: blank modal when using browser navigation

fix: duplicated results tables via JS search

fix: panic rendering results

fix: incorrect formatting of options results headers

style: remove vertical scrollbar on firefox-dev

refactor: use encoding/xml for OpenSearchDescription

refactor: switch to templ for HTML templates

build: use modd instead of watchexec

style: just use system fonts

chore: update dependencies

chore: release 0.1.6 v0.1.6

style: increase information density

chore: update simple-css

feat: add low-memory mode

refactor: use a struct to configure index options

feat: enable configuration of flags as environment variables

fix: create index metadata when index exists and meta doesn't

build: add emacs formatting configuration

feat: display results in a table, showing details on click

feat: add detail pages for packages/options

chore: reduce tracing sample rate

feat: use stale-while-revalidate instead of immutable asset paths

feat: serve assets via immutable paths

fix: moving through search history shows no results

build: re-enable asset reloading in dev

feat: make security headers stricter

feat: enable sub-resource integrity for assets

refactor: extract default config to own file

docs: move and clarify (achieved!) relevancy goal to "Goals"

fix: nil pointer exception when (re-)indexing

chore: release 0.1.5 v0.1.5

build: add command to generate changelog

feat: improve search relevance by boosting keyword matches

fix: missing alternate forms of packages with the same name

fix: incorrect (switched) behaviour of --update/--replace

fix: re-enable exit --update/--replace

chore: update dependencies

build: add task to update dependencies

build: fix flake error

feat: support cleartext HTTP/2 (i.e. from a reverse proxy)

refactor: don't log request Host, it's irrelevant to this server

build: just use nix version as internal version

build: enable non-flake import

refactor: split server cmd and module

fix: check for nil pointers in NewMux

chore: release v0.1.4 v0.1.4

refactor: use a single value for per-source import timeouts

feat: stream files directly from fetcher to importer

fix: abort import of source on batch processing errors

refactor: clarify log message when starting import

fix(web): fetch host from request, not header, where it's expected

docs: expand readme, move dev docs to subfolder

docs: document how to run an instance

build: fix pre-commit check exclude/ignore patterns

fix(nix): generate default baseURL from listen address and port

refactor(nix): rename module attribute to allow for other modules

build: narrow overly-broad linting exclusions

fix: make fetcher check on creation that it supports Source.Importer

fix(importer): abort import of current source in case of error

perf: make channel-nixpkgs importer only fetch necessary files

refactor: make repository URLs by Repo.Type (currently no change)

refactor: use enum for Repository.Type

refactor: make declaration/definition URLs from Source.Repo

build: fix `prepare` task not working after flake migration

feat: add --update flag to force-update index and exit

style: use double-dash arguments to wgo

docs: clarify help text for --replace

docs: add comments to default configuration

feat(config): use enums' string representations to print defaults

docs: add link to default config file in nix module

build: check defaults.toml wasn't modified pre-commit

fix: nix cannot read date/time TOML values from default config

feat(config): print durations and URLs with human values

build: store default configuration in repo and ensure freshness

feat: show default configuration with --print-default-config

fix: duplicate error logging on config parse error

chore: release 0.1.3 v0.1.3

feat(ui): enable resetting form by clicking current source link

feat: render markdown in packages' LongDescription

feat: use attribute instead of name for packages

fix(css): pagination styles not applied v0.1.2

fix(nix): missing version variable in build v0.1.1

chore(release): release 0.1.0 v0.1.0

build: enable versioning of package

chore: add convco configuration

feat: display more information about packages

fix: render errors as HTML given correct content-type

fix: missing last page of search results

fix: store only data from current entity in index

fix: nil pointer when updating all sources

feat: error on non-matching configuration entries

refactor: reduce debug logging

docs: update default sources in nixos module

feat: automatically add/remove sources to/from index

fix: don't panic when re-indexing if fetcher did not update

build: make `just dev` watch config.toml

feat: support searching packages

feat(index): enable storage of package data

fix(css): support browsers without css nesting

feat: improve accessibility and browser behaviour

feat: exit after rebuilding index if `--replace` is specified

refactor: export functions for parsing enum types

perf: flush index batch in groups of 10000

fix(logging): track whether headers are written properly

feat: decode brotli-compressed files

refactor: move general results template code into results template

refactor: separate fetch and import logic

fix: rebuild index when --replace is specified

refactor: remove unnecessary and incorrect bleve config option

feat: extract and store package set (e.g. nodePackages)

refactor: use interface to unify Options and Packages

refactor: remove unused interface

feat(importer): process packages from nixpkgs package.json

refactor: remove toml special cases

refactor(importer): move functions to utils file

feat: add caching header (except on error pages)

feat: monitor import jobs via sentry

fix: bad git conflict resolution

feat: add browser search engines via opensearch description

style: use consistent naming for config value

feat: print clickable URL on startup in dev mode

refactor: remove log prefix

refactor: default to development Environment outside nix module

feat: render HTML error pages

refactor(templates): render partials with less hackiness

build: remove duplicate debug log on dev file watcher update

perf: move scripts to head to enable fetching in parallel

build: bake git hash into build for headers and --version

build: use a nixpkgs-approved version scheme

build: improve --help output

fix(flake): remove overlay export

feat(module): add default for Web.Environment

fix(importer): don't create duplicate nested directories

feat: add DownloadOptions importer (fetches pre-built options.json)

feat: log on startup if index doesn't exist

refactor(importer): add context and reduce log level

feat: rely on system to show log times in production

fix(css): barely-visible text in dark mode

fix(module): include nix in path

style: make port a string for easier nix integration

fix(config): show read URL on parse failure

refactor: combine import and web server into one binary

build: fix wrong arguments in justfile

refactor(config): simplify configuration

refactor: move Source/Repository types to config package

fix(module): allow setting sentry DSN

build: workaround nix weirdness

build: export nixos module and overlay

build: switch to flakes

feat: allow adding extra HTML to the body via configuration

feat: enable using channel URLs if unable to resolve via NIX_PATH

build: limit files passed to nix

fix: don't panic on TOML decode error

refactor: move Source type

fix: linking to wrong declaration URLs

feat: nix module

feat: clicking another source re-uses query

refactor: merge search and results pages

refactor: extract index deletion function

fix(import): don't remove index if there are errors

build: add extra command to index without replacing existing index

feat(js): save and restore input state with history

fix(js): make reload after error execute attempted search

refactor(js): use async/await, it's 2024

build: optimise and fix handing of csp headers in dev mode

fix: js error when no results found

refactor: split http.Server and http.ServeMux creation

build: make `just dev` reload on go file changes

refactor: don't explain searches

fix: missing menu on search pages

style: remove extraneous <p> tags around option descriptions

fix: no results for home-manager search

build: ignore local configuration file

fix(config): add default channel name for home-manager

feat: version search index mapping schema and warn if outdated

refactor: move sources input data to $data_root/sources

refactor: deduce index path automatically from config.DataPath

refactor: split up NewIndexer function

build: enable building with nix with embedded frontend

build: optionally embed frontend files

style: highlight current search menu item

fix: missing menu on search page

style: use title-cased source names

docs: change links to point to sourcehut

refactor: clean up flags and environment variables

build: add reindex command

build: pass config file to dev command

fix: pointer type mismatch

feat: dynamically link to sources

feat: format server listen address as URL

refactor: pass Source object to templates

style: rename Option.Option to Option.Name

style: reformat with golines and enforce on commit

feat: enable index path to be configured

feat(importer): make timeout configurable via command-line

feat(importer): add --replace flag to overwrite existing index

feat: allow configuration of log level

feat: enable setting config file path via command line

docs: embed defaults in app

refactor: make config entries kebab-case

feat: import sources from configuration in go code and index options

feat: make configuration optional

fix(js): pushstate also on errors, so that the user can refresh

refactor: enable access to bleve DocumentMatch structs

refactor: reduce new variables when looping over hits

refactor: store searched documents directly in the search index

refactor: log index loading progress at startup

feat: improve search by using a custom tokeniser for option names

fix(js): null value error for pagination

fix: incorrect default/example values stored after processing

style: fix overflowing <pre> tags

fix: incorrect data shown for default and example option keys

feat: paginate search results

perf: create search indexes in parallel with server startup

feat: search multiple sources

style: show pages for different sources

style: make page wider and header shorter

style: add more text to homepage and footer

style: split homepage and search page

feat: search one set of options

build: workaround wgo CPU usage

refactor: rely on html/template functionality more

refactor: extract templating into own file

refactor: make data path (partially) configurable

fix: panic on importing darwin options

feat: link to exact commits in NixOS/nixpkgs

feat: render markdown examples

build: fetch darwin and home-manager options

feat: link to nixpkgs on github for declarations

fix: use options exposed via user's <nixpkgs> channel instead of ours

feat: auto-linkify URLs in markdown

style: increase contrast

style: remove border-radius

style: improve layout and formatting of code samples

style: add a footer

feat: render markdown in option descriptions

refactor: only reload templates when such files are updated

style: remove unused variable

style: add a small gap between search bar and button

fix(css): first grid row too tall

feat: browser history management

chore: watch all frontend files

feat: show metadata in grid

feat: frontend search implementation

chore: remove unnecessary shell variable

chore: use prettier with plugin to format go templates

feat: create nixos options.json importer

feat: add basic input frontend

chore: revert to blocklist-based gitignore

feat: add css

feat: watch and live reload in development

feat: serve a very basic html template

chore: initial commit