Turris repository builder

This implements simple script that alows you to build additional packages for Turris routers. Idea is that you can use this to create either permanent or testing builds.

You have to use machine with amd64 architecture and GNU and Linux base distribution because this tool uses precompiled OpenWRT SDK that was compiled on such machine and won't work on any other architecture. Also you need dependencies as for standard OpenWRT build (those can be found in OpenWRT documentation). On top of that a ccache was used on original build machine so you have to have it installed too.

Before first usage

Before first usage you have to create package that is going to define your repository. This will give you an easy way to install your repository on router. Let's create package myrepo for this (of course you should name it some more appropriate way). First create directory names myrepo and then Makefile file inside it. That Makefile should contain something like this:

include $(TOPDIR)/rules.mk

PKG_NAME:=<YOUR_REPO_NAME>
PKG_VERSION:=1
PKG_MAINTAINER:=<YOUR FULL NAME AND EMAIL>
PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME)

include $(INCLUDE_DIR)/package.mk

define Package/myrepo
    TITLE:=$(PKG_NAME)
endef

define Build/Compile
    :
endef

define Package/$(PKG_NAME)/install
    $(INSTALL_DIR) $(1)/etc/updater/keys
    $(INSTALL_DATA) ./files/key.pub $(1)/etc/updater/keys/$(PKG_NAME).pub

    $(INSTALL_DIR) $(1)/etc/updater/conf.d
    $(INSTALL_CONF) ./files/updater.lua $(1)/etc/updater/conf.d/$(PKG_NAME).lua
endef

$(eval $(call BuildPackage,$(PKG_NAME)))

We will return to this Makefile later on. Now we need to create directory files containing public key and configuration for updater-ng. Let's start with updater's configuration. We want to write it to file named updater.lua because that is name we have in Makefile. The content should be something like this:

local board
if model:match('[Oo]mnia') then
    board = "omnia"
else
    board = "turris"
end
Repository("<YOUR_REPO_NAME>", "https://unknown.com/" .. board .. "/", {
    pubkey = "file:///etc/updater/keys/<OUR_REPO_NAME>.pub"
})
Install("<YOUR_REPO_NAME>")
Package("<YOUR_REPO_NAME>", { replan = "finished" })

Of course replace <OUR_REPO_NAME> with your chosen name and unknown.com with real address where you plan host your repository. This tells updater where it can found your repository. There is possibility that you want to disable some security features such as OCSP or specify CRL file. For that you should look at updater's documentation.

Now let's look to keys. For simplicity you can use generate_keys.sh script that is in repository it self. It uses program called usign and to compile it you need cmake on top of OpenWRT SDK dependencies. This script will generate two files for you: key.pub and key.sec. Move key.pub to files directory of your repository package and let key.sec in current working directory.

Now you can go back to Makefile. Make sure that all <YOUR_REPO_NAME> strings are correctly replaced with name you desire and that you have filled in your name and email as a maintainer of that package.

Latest thing that you want optionally do is to create your own script calling build_repo.sh script and then immediately uploading repository to your server.

Now you should be ready to do first testing build. This build should build package you have just created. You can run build_repo.sh script in directory where you have directory with your new repository package and with key.sec file. If everything went well then you should have directory named repo in your working directory containing opkg repository with your new package.

todo: We should probably also configure opkg.

Usage

To build some package you have to place it to directory where you have your key.sec. It can't be in some subdirectory because script build_repo.sh handles only top level directories that contains file Makefile as packages. When you have it there then you can just run build_repo.sh in that top level directory. When script exits you should have directory repo with your new repository.

If build for some reason failed then you can run build_repo.sh with arguments -j1 V=99 to see exactly what went wrong.

In default build_repo.sh builds repository for Turris Omnia. You can specify turris as first argument (has to be before -j1 for example) to build packages for Turris 1.x. Or you can specify all to build both. Just to completeness script also accepts omnia as first argument to build for Turris Omnia (default behavior).

If you need some changes to configuration (such as enable/disable some additional features for OpenWRT) then you can also create file CONFIG in top level directory of your myrepo directory. Content of this file is appended to .config in sdk and that way is used during build. But be aware that using CONFIG can potentially cause rebuild of packages you haven't provided and you can end up with dependency nightmare.

Always building latest commit

As part of this repository is also a highly experimental pkgauto.mk extension. This is deployed to common include paths of packages and allows you to always build top most commit in specified branch. This is very handy for development purposes.

Only supported protocol is git.

To use this you have to first define PKG_NAME, PKG_SOURCE_URL and PKG_SOURCE_BRANCH variables (same as for any other OpenWRT package). Afterwards you have to include file $(INCLUDE_DIR)/pkgauto.mk. This file will clone repository specified by PKG_SOURCE_URL to directory .pkgauto in current working directory. From this the variables PKG_SOURCE_VERSION, PKG_VERSION and PKG_RELEASE are deduced as follows:

PKG_SOURCE_VERSION: This is filed in with hash of latest commit from PKG_SOURCE_BRANCH branch.
PKG_VERSION: This look for latest version tag on specified branch. Version tags are understood as all tags starting with v. v is cut off and rest is considered as base version for this package. To ensure that this package will always be preferred no matter on PKG_RELEASE it also appends .9999. If it locates no such commit then it uses 9999 as PKG_VERSION and fills PKG_RELEASE with number of commit since initial commit.
PKG_RELEASE: This is fill with number of commit since PKG_VERSION. To understand this more read description for PKG_VERSION.

On top of these variables it also defines some less dynamic variables. Following variables are defined with specified values:

PKG_SOURCE:=$(PKG_NAME)-$(PKG_SOURCE_VERSION).tar.gz
PKG_SOURCE_SUBDIR:=$(PKG_NAME)
PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME)
HOST_BUILD_DIR:=$(BUILD_DIR_HOST)/$(PKG_NAME)

To give some example here is head of python3-periphery package Makefile with explicit version:

include $(TOPDIR)/rules.mk

PKG_NAME:=python3-periphery
PKG_VERSION:=v1.1.0
PKG_RELEASE:=1
PKG_LICENSE:=MIT

PKG_SOURCE_PROTO:=git
PKG_SOURCE_URL:=https://github.com/vsergeev/python-periphery.git
PKG_SOURCE_VERSION:=$(PKG_VERSION)
PKG_SOURCE_SUBDIR:=$(PKG_NAME)-$(PKG_VERSION)
PKG_SOURCE:=$(PKG_NAME)-$(PKG_SOURCE_VERSION).tar.gz
PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME)-$(PKG_VERSION)

include $(INCLUDE_DIR)/package.mk
$(call include_mk, python3-package.mk)

And here is same package with pkgauto.mk used:

include $(TOPDIR)/rules.mk

PKG_NAME:=python3-periphery
PKG_LICENSE:=MIT

PKG_SOURCE_PROTO:=git
PKG_SOURCE_URL:=https://github.com/vsergeev/python-periphery.git
PKG_SOURCE_BRANCH:=HEAD

include $(INCLUDE_DIR)/pkgauto.mk
include $(INCLUDE_DIR)/package.mk
$(call include_mk, python3-package.mk)