I work in an offline development environment, which has many challenges as so much software just assumes it will be connected at all times to the Internet.
For software development on an offline network it is very handy to have locally hosted mirrors of various software repositories. Some of these are as easy as just using rsync, while others provide tools to create a mirror.
I recently needed to create an offline mirror of Hex (the Elixir language package repository). There did not appear to be any official way to create a mirror, and the few solutions already out there were dated and no longer seemed to work.
After some poking around and experimentation I realised it wouldn’t be that hard to write something that would do it. The main thing was to get the list of packages. And fortunately there was an API for that.
So I wrote a small Python script that would download the repo.hex.pm repository for offline mirroring. On subsequent runs the script will only download new files, so it is very quick.
As of June 2022 the whole repo is under 9 GB, so it doesn’t take very long to download the entire thing. The script uses parallel download jobs to speed it up.
The script is available on GitHub at https://github.com/WaterJuice/download-hexpm
To download simply run
./download-hex.pm download This will download the entire repo to a directory ./repo.hex.pm
The best way to run a mirror on an offline network is to host a webserver serving the files from the download directory as https://repo.hex.pm
In order allow hex to accept the TLS certificate either set environment variable `HEX_CACERTS_PATH` to a PEM file containing your root CA. Or configure hex with `mix hex.config cacerts_path`
Alternatively it can be served on a different address and just use HTTP. In which case set HEX_MIRROR` to point to the server. Or configure hex with `mix hex.config mirror_url`
To test on local computer you can use
python3 -m http.server ./repo.hex.pm 8000Then set environment variable HEX_MIRROR = http://localhost:8000
Water Juice 