diff options
| -rw-r--r-- | README.adoc | 41 | ||||
| -rw-r--r-- | README.md | 210 | ||||
| -rw-r--r-- | docs/README.adoc | 27 | ||||
| -rw-r--r-- | docs/build-tarball.adoc | 15 | ||||
| -rw-r--r-- | docs/initial-access.adoc | 38 | ||||
| -rw-r--r-- | docs/install-mox.adoc | 147 | ||||
| -rw-r--r-- | docs/install-omnia.adoc | 181 | ||||
| -rw-r--r-- | docs/nixos-rebuild-onsite.adoc | 44 | ||||
| -rw-r--r-- | docs/nixos-rebuild-remote.adoc | 39 | ||||
| -rw-r--r-- | docs/qemu-vs-cross.adoc | 48 | ||||
| -rw-r--r-- | docs/router.adoc | 3 | ||||
| -rw-r--r-- | docs/switch.adoc | 90 | ||||
| -rw-r--r-- | docs/todo.adoc | 60 | ||||
| -rw-r--r-- | docs/wifi-ap.adoc | 3 |
14 files changed, 736 insertions, 210 deletions
diff --git a/README.adoc b/README.adoc new file mode 100644 index 0000000..52cb298 --- /dev/null +++ b/README.adoc @@ -0,0 +1,41 @@ += NixOS for Turris routers + +This repository contains nix flake to get NixOS running on Turris routers. + +Using NixOS on router has few shortcomings, and you should know about them right +away so here is a list of issues you should expect: + +* NixOS is pretty memory hungry. There are few reasons for this such as Systemd + and specially Journald as those simply required more memory than other + alternatives. But but but... The biggest issue is NixOS itself. The wide range + of configuration options in NixOS result in pretty significant memory being + required on evaluation. The peak is around 2 GB and thus device with 512 MB is + pretty much screwed. Thankfully Nix is not using that memory most of the time, + and thus we can use swap without hampering the performance that much. +* Firewall configuration right now expects at most one NAT network thus forget + about multiple networks. There can't be dedicated network for you IOT devices + or for your guests. +* Hostapd configuration is complex and it is pretty hard to configure functional + Wi-Fi with higher standards than 802.11b. +* VLAN filtering on the switch is not easilly configured and thus you have to + use multiple bridges to separate networks. This also means that having + untagged traffic on the same trunk as tagged one is not easilly possible + unless you accept that bridge that contains untagged interface distributes all + traffic including the tagged one. + +This repository provides the following functionality on top of the standard +NixOS: + +* Minimal system configuration to support Turris Omnia and Mox. +* Simple module for LEDs configurations for Turris Omnia and Mox. +* Improved Hostapd configuration as the default one in NixOS supports only + single wireless network and adapter. +* Simple way to cross build NixOS with path: + `config.system.build.cross.${system}.config.system.build.toplevel` +* Simple way to build tarball to deploy NixOS to Turris: + `config.system.built.tarball` + +IMPORTANT: This repository required Nix with flakes support thus update your Nix +to the latest version and allow flakes. + +For more info feel free to read ./docs/README.adoc[documentation]. diff --git a/README.md b/README.md deleted file mode 100644 index 36a0199..0000000 --- a/README.md +++ /dev/null @@ -1,210 +0,0 @@ -# NixOS for Turris routers - -This repository contains nix flake to get NixOS running on Turris routers. - -Using NixOS on router has few shortcomings, and you should know about them right -away so here is a list of issues you should expect: - -* NixOS is pretty memory hungry. There are few reasons for this such as Systemd - and specially Journald as those simply required more memory than other - alternatives. But but but... The biggest issue is NixOS itself. The wide range - of configuration options in NixOS result in pretty significant memory being - required on evaluation. The peak is around 2 GB and thus device with 512 MB is - pretty much screwed. Thankfully Nix is not using that memory most of the time, - and thus we can use swap without hampering the performance that much. -* Firewall configuration right now expects at most one NAT network thus forget - about multiple networks. There can't be dedicated network for you IOT devices - or for your guests. -* Hostapd configuration is pretty stupid and is prepared pretty much only for - single Wi-Fi card with single wireless network. Forget about multiple Wi-Fi - networks. - -**Warning**: This repository required Nix with flakes support thus update your -Nix to the latest version and allow flakes. - - -## Turris Mox - -Deploying NixOS to the Turris Mox is pretty easy and fast. The advantage is that -aarch64 packages cache is provided by official Hydra builds. - -At minimum, you should use micro SD card with size 8 GB. It is highly suggested -to use some high quality one as we have to use it for swap and that can reduce -lifetime a lot and kill the low quality one very fast. - -The updated U-Boot is required to make Turris Mox work with NixOS. This can be -done on Turris OS 6.0 with command `turris-nor-update`. The second modification -that is required is change of ramdisk load address (see the next section). - -### Prepare the SD card - -You need to format the SD card first. The GPT is suggested as the partition -table. You can use only one partition that should be formatted with BTRFS. The -important thing is to set label `NixTurris` so boot can locate the correct -parition by it. - -It is up to you which tool are you going to use. The instructions here are going -to be for GNU Parted. The following shell log should give you an idea of what -you should do to get the correct layout: - -``` -~# parted /dev/mmcblk1 -(parted) mktable gpt -(parted) mkpart NixTurris 0% 100% -(parted) set 1 boot on -(parted) quit -~# mkfs.btrfs /dev/mmcblk1p1 -``` - -Next we need the initial system tarball to unpack to the SD card. Add nixturris -repository to your local Nix registry and build it. The image is cross compiled -in this case (unless you are running on Aarch64 platform). You can also build it -natively and this is discussed in the chapter "Native build using Qemu" in this -document. - -``` -~$ nix registry add nixturris git+https://git.cynerd.cz/nixturris -~$ nix build nixturris#crossTarballMox -``` - -The last step is to unpack the tarball to the SD card. - -``` -~# mkdir -p mnt -~# mount /dev/mmcblk1p1 mnt -~# tar -xf result/tarball/nixos-system-aarch64-linux.tar.xz -C mnt -~# umount mnt -~# eject /dev/mmcblk1 -``` - -Now you can take this micro SD card and insert it to your Mox. Before you start -it you should also check the following chapter as it won't most likely boot -unless you modify the default boot environment. - -### System fails to boot due to invalid initrd - -The issue is caused by initrd start being overwritten by kernel image's tail. - -The kernel image in NixOS can be pretty large and default Mox's configuration -expects kernel of maximum size 48MB. To increase this to 64MB you have to use -serial console and run: - -``` -setenv ramdisk_addr_r 0x9000000 -saveenv -``` - - -## Turris Omnia - -### Botting from the USB - -Requires updated U-Boot! - -``` -run usb_boot - -setenv boot_targets usb0 mmc0 nvme0 scsi0 pxe dhcp -saveenv -boot -``` - -## Updating / rebuilding NixOS and pushing update - -The suggested way to update NixOS on Turris is to build system on more powerful -device and push only resulting build to the device. The reason for this are -memory requirements for Nix itself. NixOS and its packages repository evaluation -consumes a lot of memory and thus doing that on different PC is just faster. - -Prepare directory where you are going to be managing your Turris device(s) and -paste this `flake.nix` file in it: - -``` -{ - description = "Turris system management flake"; - - inputs = { - nixturris = { - url = "git+https://git.cynerd.cz/nixturris"; - inputs.nixpkgs.follows = "nixpkgs"; - }; - }; - - outputs = { self, nixpkgs, flake-utils, nixturris }: - with flake-utils.lib; - { - nixosConfigurations = let - - turrisSystem = board: hostname: { - ${hostname} = nixturris.lib.nixturrisSystem { - nixpkgs = nixpkgs; - board = board; - modules = [ - # Place for your modules - ]; - }; - }; - - in - turrisSystem "mox" "moxhost" // - turrisSystem "omnia" "omniahost"; - - }; -} -``` - -**TODO** describe here how to generate key, sign build and distribute it to the -device. - -To update system before build run `nix flake update`. To update system - - -## Updating / rebuilding NixOS on device - -To rebuild NixOS directly on device something like 2GB of memory is required. -This pretty much is covered only by Turris Omnia 2G version and even that might -not be enough. Thus if you want to rebuild NixOS on the device you need the -swap. There is in default configure zram swap but that won't be enough. It is -highly suggested to create swap file of something like 2GB or 4GB size. - -The creation and first addition of swap can be done like this (this expects that -used file-system is BTRFS): - -``` -sudo truncate -s 4G /run/swap -sudo chattr +C /run/swap -sudo btrfs property set /run/swap compression none -sudo chmod 600 /run/swap -sudo mkswap /run/swap -sudo swapnon -p 0 /run/swap -``` - -Few notes here... Swap file is created by root and set to be accessible only by -root. For BTRFS the copy-on-write functionality is disabled and compression for -it. The swap itself is then added with lowest priority to prefer zram swap and -thus reduce real swap usage. - -Do not forget to add this swap file to your NixOS configuration so it is added -on every boot. - -Then you can pretty much manage it as any other NixOS device using -`nixos-rebuild` script running directly on the device, just very slowly. - - -## Native build using Qemu - -This document references cross compilation in default but there are good reasons -for not using it. It can be broken much more often for some packages. It also -requires complete rebuild when later updating natively on the platform as cross -build is just not compatible with native build in Nix. - -To get native build work you need the Nix with ability to run aarch64 binaries. -That is in general option `extra-platform` with value `aarch64-linux` for Turris -Mox and `armv7l-linux` for Turris Omnia. To actually allow access to the -emulator you need something like `extra-sandbox-paths` with value such as -`/usr/bin/qemu-aarch64` (for Turris Mox) or `/usr/bin/qemu-arm` (for Turris -Omnia). This way you can build aarch64 or armv7l package on other platforms. -With this setup you should be able to build tarball natively and instead of -`.#crossTarball*` you can now use just `.#tarbal*`. - -## Custom tarball or system build diff --git a/docs/README.adoc b/docs/README.adoc new file mode 100644 index 0000000..6a264a4 --- /dev/null +++ b/docs/README.adoc @@ -0,0 +1,27 @@ += NixOS for Turris routers documentation + +Please see general remarks about this project in the top level +../README.adoc[README file]. + +To start with NixOS on Turris please read install documentation for your +specific board: + +* ./install-omnia.adoc[*Omnia installation*] +* ./install-mox.adoc[*Mox installation*] + +The board specific instructions provide you with hopefully working installation +of the NixOS on Turris OS. The following articles should give you more info on +how to manage it: + +* ./initial-access.adoc[*Initial router access*] +* ./nixos-rebuild-remote.adoc[*NixOS rebuild remotely and pushing changes*] +* ./nixos-rebuild-onsite.adoc[*NixOS rebuild on the router*] +* ./switch.adoc[*Switch setup*] +* ./router.adoc[*Router setup*] +* ./wifi-ap.adoc[*Wi-Fi Access Point*] + +Some additional topics: + +* ./qemu-build.adoc[*Cross build vs Qemu build*] +* ./build-tarball.adoc[*Building system tarball*] +* ./todo.adoc[*Things to be done and improved*] diff --git a/docs/build-tarball.adoc b/docs/build-tarball.adoc new file mode 100644 index 0000000..b5f5fb5 --- /dev/null +++ b/docs/build-tarball.adoc @@ -0,0 +1,15 @@ += NixTurris tarball + +TODO + +== Building standard tarball + +Add nixturris repository to your local Nix registry and build it. The image is +cross compiled in this case (unless you are running on Aarch64 platform). You +can also build it natively and this is discussed in the chapter "Native build +using Qemu" in this document. + +``` +~$ nix registry add nixturris git+https://git.cynerd.cz/nixturris +~$ nix build nixturris#crossTarballMox +``` diff --git a/docs/initial-access.adoc b/docs/initial-access.adoc new file mode 100644 index 0000000..e6d67f8 --- /dev/null +++ b/docs/initial-access.adoc @@ -0,0 +1,38 @@ += Initial router access after installation + +This describes how to access the router right after the initial NixOS +installation. The WAN port has to be connected to the existing network to receive +IP address. This has to be the same network the machine you are planning on +using for connecting to the board is connected to. You have to get this IP +somehow. One option is to use https://docs.turris.cz/hw/serial/[serial +connection] to get it by logging in and entering command `ip addr`. Another +option is scanning the local network or if you have access you can just look for +`NixTurris` system in DHCP logs. + +Once you have IP address you should be able to login over SSH. The SSH allows +loging to root user with password `nixturris`. + +Make sure you setup your own access route to the device before you do the +initial rebuild as this setup is set only for the tarball boot and is removed +with first boot. Feel free to paste the following to your initial configuration +so you would not lost this default setting initialy: + +---- +users = { + mutableUsers = false; + users.root.password = mkDefault "nixturris"; +}; +services.openssh = { + enable = true; + passwordAuthentication = true; + permitRootLogin = "yes"; +}; +---- + +Make sure you change this as soon as possible as this is highly unsafe. + +Now you can follow to the first NixOS rebuild. There you have to decide if you +prefer ./nixos-rebuild-remote.adoc[managing it remotely] or +./nixos-rebuild-onsite.adoc[managing it directly on the router]. The remote way +is more complex to setup but much faster in the long run while rebuild on the +router can be pretty much done immediately. diff --git a/docs/install-mox.adoc b/docs/install-mox.adoc new file mode 100644 index 0000000..56c0e31 --- /dev/null +++ b/docs/install-mox.adoc @@ -0,0 +1,147 @@ += NixOS on Turris Mox + +Deploying NixOS to the Turris Mox is pretty easy and fast. The advantage is that +aarch64 packages cache is provided by official Hydra builds and thus there is no +need to build everything from the sources. + +TIP: Read the whole documentation before you start to understand what you are +getting yourself into. + +NOTE: The Turris Shield is in core Turris Mox and thus these instructions can be +used for it as well. + + +== 1.) Updating U-Boot + +The updated U-Boot is required to make Turris Mox work with NixOS. The reason +for this is limited small length for the string holding path to the files for +system boot in the U-Boot version 2018.11 (the one Turris Mox ships with). + +You can verify if you have new enough U-Boot by running the following command as +root on the router: + +---- +strings /dev/mtd1 | grep 'U-Boot [0-9.]\+' +---- + +The minimal suggested version is U-Boot 2022.07. + +=== Updating U-Boot on Turris OS + +These instructions apply if you have SD card with Turris OS and want to use the +official way of updating firmware on Turris Mox. The minimal version of Turris +OS to get new stable version of U-Boot is Turris OS 6.0. Please update the +Turris OS installation to this version or newer before you continue. + +The next step is to install `turris-nor-update` and run it: + +---- +opkg update +opkg install turris-nor-update +turris-nor-update +---- + +This should give you at least U-Boot 2022.07. + +=== Updating U-Boot on NixOS + +TODO + + +== 2.) Expanding space for Linux kernel + +The space allocated for the kernel during the boot is in default set to be too +small. The kernel of NixOS can get pretty big compared to the Turris OS and +thus we have to increase this space. + +CAUTION: The system is going to fail to boot with cryptic message that it can't +mount root file-system if you skip this! That is caused by initrd start being +overwritten by kernel image's tail. + +=== Directly from U-Boot + +You need compatible serial to USB converter to be attached to the serial console +of your Mox. Consult the https://docs.turris.cz/hw/serial/#turris-mox[official +documentation] for this. + +The first step is to enter U-Boot console, reboot your router (either by issuing +`reboot` command or by disconnecting and reconnecting the power source). Next +press kbd:[Enter] few times once you see `U-Boot` booting. You should get U-Boot +prompt shortly after (the line starting with `> `). Enter the following commands +to set required settings: + +---- +setenv ramdisk_addr_r 0x9000000 +saveenv +---- + +To continue boot you can enter command `boot`. + +=== From running system + +You need to have `uboot-envtools` package installed on Turris OS or `ubootTools` +on NixOS. + +Verify that your environment access is correctly set by printing a current +environment using `fw_printenv` (as root). You should get output with no message +informing you about invalid CRC. Do not proceed if you get it. + +To set required settings run: + +---- +fw_setenv ramdisk_addr_r 0x9000000 +---- + +Feel free to validate settings by running `fw_printenv` again. + + +== 3.) Get tarball with NixOS + +The initial system can either be downloaded from Gitlab or build locally. Note +that version available for download can be pretty old. It is just some version +that was build with latest commit to the NixTurris repository. + +* TODO (we do not have CI builds yet) +* ./build-tarball.adoc[Documentation on how to build your own tarball] + + +== 4.) Prepare the SD card + +At minimum, you should use micro SD card with size 8 GB. It is highly suggested +to use some high quality one as you might have to use it for swap and that can +reduce lifetime a lot and kill the low quality one very fast. + +You need to format the SD card first. The required partition table is GPT. You +can use only one partition that would be preferably formatted with BTRFS. The +important thing is to set label `NixTurris` so boot can locate the correct +partition by it. + +It is up to you which tool are you going to use. The instructions here are going +to be for GNU Parted. The following shell log should give you an idea of what +you should do to get the correct layout: + +---- +~# parted /dev/mmcblk1 +(parted) mktable gpt +(parted) mkpart NixTurris 0% 100% +(parted) set 1 boot on +(parted) quit +~# mkfs.btrfs /dev/mmcblk1p1 +---- + +The last step is to unpack the tarball to the SD card. + +---- +~# mkdir -p mnt +~# mount /dev/mmcblk1p1 mnt +~# tar -xf nixos-system-aarch64-linux.tar.xz -C mnt +~# umount mnt +~# eject /dev/mmcblk1 +---- + +Now you can take this micro SD card and insert it to your Mox and boot the +initial version of the system. + +Now you should have running NixOS system on the Turris Mox. The next step is to +do first configuration and for that you have to be able to +./initial-access.adoc[access the router]. diff --git a/docs/install-omnia.adoc b/docs/install-omnia.adoc new file mode 100644 index 0000000..4c5abac --- /dev/null +++ b/docs/install-omnia.adoc @@ -0,0 +1,181 @@ += NixOS on Turris Omnia + +CAUTION: PCIe devices are not functional right now on the Turris Omnia! + +The Turris Omnia is little bit more complex regarding the initial setup compared +to Turris Mox. One issue is that armv7 builds are not provided by official +Hydra. The second issue is that there is not a single reasonable installation +(such as SD card on Turris Mox). + +Turris Omnia can be booted either from USB or internal disk or from internal +MMC. The USB drive is the easies to setup from PCs and thus it is suggested for +testing NixOS on your Turris Omnia. You can always return to the Turris OS just +by unplugging the USB drive. + +== 1.) Updating U-Boot + +The updated U-Boot is required to make Turris Omnia work with NixOS in most +cases. The reason is that the original U-Boot 2015.10 does not support booting +from USB or internal drive. The newer Turris Omnia routers have newer U-Boot +versions but always verify the U-Boot version before you continue by running the +following command as root on the router: + +include::shards/install-uboot.adoc[] + +---- +strings /dev/mtd0 | grep 'U-Boot [0-9.]\+' +---- + +The minimal suggested version is U-Boot 2022.07. + +=== Updating U-Boot on Turris OS + +These instructions apply if you have SD card with Turris OS and want to use the +official way of updating firmware on Turris Mox. The minimal version of Turris +OS to get new stable version of U-Boot is Turris OS 6.0. Please update the +Turris OS installation to this version or newer before you continue. + +The next step is to install `turris-nor-update` and run it: + +---- +opkg update +opkg install turris-nor-update +turris-nor-update +---- + +This should give you at least U-Boot 2022.07. + +=== Updating U-Boot on NixOS + +TODO + +== 2.) Get tarball with NixOS + +The initial system can either be downloaded from Gitlab or build locally. Note +that version available for download can be pretty old. It is just some version +that was build with latest commit to the NixTurris repository. + +* TODO (we do not have CI builds yet) +* ./build-tarball.adoc[Documentation on how to build your own tarball] + +== 3a.) Boot from USB drive + +This is the easies option to get NixOS on Turris Omnia. You can prepare the +drive in any Linux system and then just plug it into the Omnia. + +You should use some reliable and preferably USB 3.0 flash drive or just regular +SSD drive. The suggested minimal size is 8GB but that is pretty much every USB +drive on the market right now. You might want to use swap and thus make sure +that you use some higher quality one as swap can kill cheap flash drives pretty +fast. + +You need to format the USB drive first. The required partition table is GPT. You +can use only one partition that would be preferably formatted with BTRFS. The +important thing is to set label `NixTurris` so boot can locate the correct +partition by it. + +It is up to you which tool are you going to use. The instructions here are going +to be for GNU Parted. The following shell log should give you an idea of what +you should do to get the correct layout: + +---- +~# parted /dev/sdx +(parted) mktable gpt +(parted) mkpart NixTurris 0% 100% +(parted) set 1 boot on +(parted) quit +~# mkfs.btrfs /dev/sdx1 +---- + +The next step is to unpack the tarball to the USB drive. + +---- +~# mkdir -p mnt +~# mount /dev/sdx1 mnt +~# tar -xf nixos-system-armv7-linux.tar.xz -C mnt +~# umount mnt +~# eject /dev/sdx +---- + +Now you can take this drive and plug it in the Turris Omnia. The last step is to +instruct firmware to boot from USB. + +TIP: If you have USB to serial converter (see +https://docs.turris.cz/hw/serial/#turris-omnia[official documentation]) then you +can trigger USB boot only once using it. You have to reboot Omnia and halt boot +process by pressing kbd:[Enter] few times once you see `U-Boot` booting. You +should get U-Boot prompt shortly after (the line starting with `> `). USB boot +is initialized by entering `run usb_boot` to it and confirming by kbd:[Enter]. + +You need to have `uboot-envtools` package installed on Turris OS or `ubootTools` +on NixOS. + +First you have to verify your environment access configuration by printing the +current environment using `fw_printenv` (as root). You should get output with no +message informing you about invalid CRC. Do not proceed if you get it. + +To set USB boot as a preferred method run: + +---- +fw_setenv boot_targets usb0 mmc0 nvme0 scsi0 pxe dhcp +---- + +Feel free to validate settings by running `fw_printenv` again. + +Now you can reboot your Omnia to NixOS. The next step is to do first +configuration and for that you have to be able to ./initial-access.adoc[access +the router]. + +== 3b.) Install on internal storage + +It is highly suggested not to use internal MMC for the whole system. The +suggested deployment is rather the `/boot` partition on MMC or the whole system +on internal drive (note that U-Boot might not be able to boot from all internal +drives). + +TODO + +WARNING: this is not fully supported at the moment as tarball contains in +`/boot` only Syslinux configuration and thus boot simply from `/boot` is not +possible. You have to copy kernel, initrd and dtb to the `/boot` partition and +modify Syslinux's configuration file. + + +== Reverting to Turris OS + +The steps required to revert to Turris OS differ from how deep have you went +with installation. If you are lucky it is just about restarting U-Boot +environment, if not then you have to use Medkit to install Turris OS back. + +=== Resetting U-Boot environment to the default + +The modification suggested by this document are only to the single variable +(`boot_targets`). The revert to the previous value should be in most cases +enough. This can be done by (see previous usage in this document for required +packages): + +---- +fw_setenv boot_targets mmc0 nvme0 scsi0 usb0 pxe dhcp +---- + +You can also reset environment to the default if you want to be sure. For that +you need USB to serial converter connected (see +https://docs.turris.cz/hw/serial/#turris-omnia[official documentation]). Reboot +the board and halt boot process by pressing kbd:[Enter] few times once you see +`U-Boot` booting. You should get U-Boot prompt shortly after (the line starting +with `> `). To reset environment enter: + +---- +env default -a +saveenv +---- + +To continue boot you can enter command `boot`. + +=== Using Medkit to install Turris OS back to the MMC + +You have to do this only if you attempted installation on internal storage. +First you have to reset U-Boot environment and thus follow steps in the previous +section. Next follow the +https://docs.turris.cz/hw/omnia/rescue-modes/#re-flash-router[official medkit +steps]. diff --git a/docs/nixos-rebuild-onsite.adoc b/docs/nixos-rebuild-onsite.adoc new file mode 100644 index 0000000..80c8583 --- /dev/null +++ b/docs/nixos-rebuild-onsite.adoc @@ -0,0 +1,44 @@ += NixOS rebuild on the router + +WARNING: This is pretty slow. + +To rebuild NixOS directly on device something like 2GB of memory is required. +This pretty much is covered only by Turris Omnia 2G version and even that might +not be enough. Thus if you want to rebuild NixOS on the device you need the +SWAP. There is in default configure zram SWAP but that won't be enough. It is +highly suggested to create swap file of something like 2GB or 4GB size. + +The creation and first addition of swap can be done like this (this expects that +used file-system is BTRFS): + +[source,sh] +---- +sudo truncate -s 4G /run/swap +sudo chattr +C /run/swap +sudo btrfs property set /run/swap compression none +sudo chmod 600 /run/swap +sudo mkswap /run/swap +sudo swapnon -p 0 /run/swap +---- + +Few notes here... Swap file is created by root and set to be accessible only by +root. For BTRFS the copy-on-write functionality is disabled and compression for +it. The swap itself is then added with lowest priority to prefer zram swap and +thus reduce real swap usage. + +Do not forget to add this swap file to your NixOS configuration so it is added +on every boot. Edit file `/etc/nixos/configuration.nix` and add the following +lines: + +[source,nix] +---- +swapDevices = [ + { + device = "/run/swap"; + priority = 0; + } +]; +---- + +Then you can pretty much manage it as any other NixOS device using +`nixos-rebuild` script running directly on the device, just very slowly. diff --git a/docs/nixos-rebuild-remote.adoc b/docs/nixos-rebuild-remote.adoc new file mode 100644 index 0000000..4794b37 --- /dev/null +++ b/docs/nixos-rebuild-remote.adoc @@ -0,0 +1,39 @@ += NixOS rebuild remotely + +The suggested way to update NixOS on Turris is to build system on more powerful +device and push only resulting build to the device. The reason for this are +memory requirements for Nix itself. NixOS and its packages repository evaluation +consumes a lot of memory and thus doing that on different PC is just faster. + +Prepare directory where you are going to be managing your Turris device(s) and +paste this `flake.nix` file in it: + +[source,nix] +---- +{ + description = "Turris system management flake"; + + inputs = { + nixturris.url = "git+https://git.cynerd.cz/nixturris"; + }; + + outputs = { self, nixpkgs, flake-utils, nixturris }: + with flake-utils.lib; + { + nixosConfigurations.turris = nixturris.lib.nixturrisSystem { + nixpkgs = nixpkgs; + board = "mox"; <1> + modules = [ + # Place for your modules + ]; + }; + }; +} +---- + +<1> Declares board type. Change to `omnia` if you are managing Turris Omnia. + +**TODO** describe here how to generate key, sign build and distribute it to the +device. + +To update system before build run `nix flake update`. diff --git a/docs/qemu-vs-cross.adoc b/docs/qemu-vs-cross.adoc new file mode 100644 index 0000000..7010ac0 --- /dev/null +++ b/docs/qemu-vs-cross.adoc @@ -0,0 +1,48 @@ += Cross build vs. Qemu build + +When building packages or NixOS on any other machine (most likely amd64) you +have two options how to build. You can either build natively like you would be +running on the target platform using Qemu. Or you can use cross build. This +document tires to gives you some comparison so you can decide on your own which +you want to use. In general use what works for you. + +Compilation speed:: The cross build is much faster compared to Qemu. It is +simple as that. + +Build success:: There is a lot of packages that are broken for cross build in +Nixpkgs and thus sometimes you might have more luck with native builds. At the +same time there is a lot of packages that won't compile natively on armv7 for +multiple reasons (the most common is that test cases do not expect 32bit +system). The rule of thumb is that with aarch64 native build is more stable +while cross build might be more stable for armv7. + +Device rebuilds:: The one huge disadvantage for the cross builds is that they +produce different result compared to native builds in Nix. The effect is that +`nixos-rebuild` on the device can't reuse cross builds while it can reuse native +builds. + + +== Native (Qemu) build Nix setup + +To get native build work you need the Nix with ability to run aarch64 and/or +armv7 binaries. That is in general option `extra-platforms` with value +`aarch64-linux` for Turris Mox and `armv7l-linux` for Turris Omnia. To actually +allow access to the emulator you need something like `extra-sandbox-paths` with +value such as `/usr/bin/qemu-aarch64` (for Turris Mox) or `/usr/bin/qemu-arm` +(for Turris Omnia). This way you can build aarch64 or armv7l package on other +platforms. You might also want to add some ARM specific substituters. + +The configuration file for Nix should thus contain something like: + +---- +extra-platforms = aarch64-linux armv7l-linux +extra-sandbox-paths = /usr/bin/qemu-aarch64 /usr/bin/qemu-arm +extra-substituters = https://arm.cachix.org https://thefloweringash-armv7.cachix.org +extra-trusted-public-keys = arm.cachix.org-1:K3XjAeWPgWkFtSS9ge5LJSLw3xgnNqyOaG7MDecmTQ8= thefloweringash-armv7.cachix.org-1:v+5yzBD2odFKeXbmC+OPWVqx4WVoIVO6UXgnSAWFtso= +---- + +If you are running on the NixOS then you can use configuration: + +---- +boot.binfmt.emulatedSystems = [ "aarch64-linux" "armv7l-linux" ]; +---- diff --git a/docs/router.adoc b/docs/router.adoc new file mode 100644 index 0000000..6c04640 --- /dev/null +++ b/docs/router.adoc @@ -0,0 +1,3 @@ += NixOS as router + +TODO diff --git a/docs/switch.adoc b/docs/switch.adoc new file mode 100644 index 0000000..10d5434 --- /dev/null +++ b/docs/switch.adoc @@ -0,0 +1,90 @@ += NixOS as switch + +The Linux system uses DSA to manage on board switch. DSA provides switch +configuration abstraction in such a way that every LAN port of the switch is +actually available for the network configuration. Thus switch configuration is +as simple as assigning the correct port to the bridge. + +NOTE: The examples shown here are for Turris Mox with 4 port switch but +modification required for Omnia and other Mox configurations should be clear +from them. + +The default single LAN setting would look something like this: + +[sources,nix] +---- +networking = { + bridges.brlan <3> = { + interfaces = [ + "eth0" "lan1" "lan2" "lan3" "lan4" <2> + ]; + }; + dhcpcd.allowInterfaces = [ "brlan" ]; <3> +}; +---- + +<1> The bridge interface name. +<2> Bridge WAN port with all LAN ports. Note that this automatically disables +DHCP on these ports in NixOS. +<3> Set DHCP server on our new bridge as it ignores bridges automatically. + +The more complex setup with VLANs might look like this. Let's consider that WAN +port is connected to the router that provides three VLANs and we want to assign +VLAN 1 to `lan1` and VLAN 2 to `lan2` while connecting `lan3` and `lan4` with +WAN. The VLAN100 is administation and switch should listen only on that network +with static IP. + +[sources,nix] +---- +networking = { + vlans = { <1> + "brlan.1" = { + id = 1; + interface = "brlan"; + }; + "brlan.2" = { + id = 2; + interface = "brlan"; + }; + "brlan.100" = { + id = 100; + interface = "brlan"; + }; + }; + bridges = { + brlan.interfaces = [ + "eth0" "lan3" "lan4" <2> + ]; + brlan1.interfaces = [ + "brlan.1" "lan1" + ]; + brlan2.interfaces = [ + "brlan.2" "lan2" + ]; + }; + interfaces."brlan.100" = { <3> + ipv4 = { + addresses = [{ + address = "192.168.100.42"; + prefixLength = 24; + }]; + }; + }; + defaultGateway = "192.168.100.1"; + nameservers = [ "192.168.100.1" "1.1.1.1" "8.8.8.8" ]; + networking.useDHCP = false; <4> +}; +---- + +<1> Create VLAN interfaces used in bridges and to actually access the router. +<2> This bridge provides inteconnection between WAN and `lan3` and `lan4` while +serving as base for our VLANs. +<3> Static IPv4 configuration for our management port. The IPv6 should be +assigned by SLAAC. +<4> Disable DHCP as we do not want access to the router from any other interface +than statically configured one. + +WARNING: The vlan filtering can't be easilly configured and obvious way of +adding `lan0.1` to `br1` and `lan0` to `br2` results in tagged traffic being +included in both `br1` and `br2`. The "correct" way of configuring the complex +example here would be by using single bridge and filtering VLANs. diff --git a/docs/todo.adoc b/docs/todo.adoc new file mode 100644 index 0000000..69b5149 --- /dev/null +++ b/docs/todo.adoc @@ -0,0 +1,60 @@ += Things to be done + +NixTurris is open-source project without any monetary backing. It is just an +attempt to get NixOS and its advantages to the Turris routers. Please poke to +these topics if you have some free time. + +These are big tasks. For the smaller ones please use Gitlab issues. + + +== Bugs to be solved + +Generate template configuration in `/etc/nixos/`:: This is common way +the installation of NixOS proceeds. We would use it only with NixOS rebuild on +site but we should still do it. + +PCIe on Omnia does not work:: There are patches that make it work on Turris OS +and attempt to port them to this repository but some important patch seems to be +still missing as it doesn't work. + +Boot files should be copied to `/boot` on tarball:: The standard syslinux +generator copies boot files (linux, initrd and dtb) to `/boot` while current +implementation in tarball keeps them in Nix store. This prevents installation on +separate boot partition. + +Native build for Turris Omnia doesn't work:: There are multiple packages that +fail to build when compiled natively on armv7. This can be either due to Qemu or +more likely that they are just not prepared to be build on armv7. The most of +the packages fail their test phases as they most likely expect 64bit system. + + +== Improvements to be tackled + +Firewall configuration with zones and masquarade:: The NixOS firewall should be +expanded or replaced by firewall configuration that provides multiple zones with +ability to set NAT between them. + +Easier Wi-Fi AP configuration:: There should be some better way to configure +Wi-Fi without reading load of documentation on how to setup Hostapd. One option +is to have minimal user configuration and fill the rest when generating +configuration based on Wi-Fi adapter. The second and probably easier option is +to just provide simple configuration with verified settings for the Wi-Fi cards +shipped with Turris routers. + +Minimal system build:: The Nix evaluation consumes considerable amount of RAM +and thus it is not possible to rebuild NixOS on the device itself without using +SWAP. The option is to cut out all modules that are just not required as they +deal with graphics output and desktop. The initial work on that is already +started in this repository but issue is that even core modules depend on desktop +specific ones. + +Nix build of U-Boot:: The initial support for this is already present but we are +missing some helper script to flash files to the appropriate block devices. The +Turris Mox firmware also requires combination with ARM Trusted Firmware which is +not done yet. + +Boot modes on Omnia to select previous profiles:: By modifying the U-Boot +environment for rescue we might be able to use Omnia standard LEDs rescue mode +selection to select older profiles. The selected led is passed just by set +environment variable to the U-Boot script but we would have to patch the U-Boot +to change the rescue environment. diff --git a/docs/wifi-ap.adoc b/docs/wifi-ap.adoc new file mode 100644 index 0000000..cfc80bf --- /dev/null +++ b/docs/wifi-ap.adoc @@ -0,0 +1,3 @@ += NixOS Wi-Fi Access Point + +TODO |