Devuan Notes for Aug 16(17) 2017
Pad:
https://pad.dyne.org/code/#/1/edit/nVuY1HnoWtgH6bmSyrWKxQ/tBEct7e9LKxhIY4Rx7I7dfJN
Present: Centurion, Evilham, fsmithred, rrq, golinux, dimitsos (pad,
irc), NewGnuGuy (observing)
Missing: The usual suspects.
## Infrastructure discussion (priority)
-
https://pad.dyne.org/code/#/1/edit/usQoFADFDsJ9UAFbVm0H7A/FPGyscaoxWTAqqgscT1bw7VA
## Build improvements discussion
-
https://pad.dyne.org/code/#/1/edit/nXiySd0FPHvG8pBgGgImxA/aJ87TdwIEWiIk96GZcFwypnb
## Centurion Dan
It feels to me like we have a number of key issues in play:
1) How to allow other distributions to use our CI system without
compromising on security? To me there are several factors to consider:
a) What benefit does hosting each of these other distributions
bring to Devuan and does that benefit offset the cost of resources
required to provide the systems including developer/admin/training time?
b) Do we allow them to cherry-pick our resources - ie use jenkins
without requiring them to use git.devuan.org (because gdo currently
provides permissions to authorise actions on jenkins)?
c) Will their use of our CI system constrain our ability to further
develop/enhance/augment our CI system?
d) How do we arbitrate potentially scarce resources (ie buildhosts)
when there is competition for those resources (think when urgent
security updates need to be built and another project has queued up a
massive number of jobs? (_Currently we don't but jenkins does have the
capability, to set constraints on a per user/group basis for number of
running builds etc but some time would be required to consider and set
this up)?
2) How to make triggering builds in the CI system easier then it is
now?
3) How to provide finer grained (ie per suite) build permissions?
4) Is there a real security vulnerability with gitlab that makes it a
now unwise to continue to use as the federated authentication
provider for our git and CI, and Devuan as a whole system?
**With respect to the scorsh proposal:
I'll agree that scorsh as designed provides a solution to some of
these stated problems, but I believe it does so at the cost of
transparancy, and adds significant complexity and ignores the bigger
picture beyond just building packages
**My concerns about scorsh based on it's DPI/RFC:**
- Scorsh has it's own user database that is not representative of the
existing user database in gitlab. This is problematic as users will
need to be administered in 2 places.
** A possible solution is that scorsh provides an OAuth integrated
website to allow users to manage their own PGP/GPG keys. **
- Scorsh adds a separate permissions system for authorisation - one
which is entirely disconnected and doesn't respect any of the existing
gitlab federated permissions. This means we will have two separate sets
of permissions to authorise:
a) who can upload to git.d.o on a project and branch level
b) who can build the package of a project for a suite
_ To me there is no reason to differentiate who can commit to
suites/jessie and who can build suites/jessie. If we don't trust
the committers commits to be built they shouldn't be directly
committed to that branch._
** A solution to this will be to make scorsh use gitlabs permissions
(available via the api) to ensure anybody that can push to a branch
in gitlab also can trigger the build of that package. This obviates
implementing the user self management of their own keys as suggested
above. **
- The Scorsh design requires a refactored version of releasebot that:
i) bypasses releasebots usual permissions model - because scorsh
implements it's own
ii) potentially requires different templates for other distributions
and thus requires adding code to select those different templates.
__+ this probably means a fork of releasebot is required, resulting
in both duplication of function and risk that templates diverge.
Thus a job add or modification in scorsh results in a different
jenkins job config to that which is provided by gitlab triggered
action.__
**Thoughts on Refactoring Releasebot to solve these problems:
Adding additional build permissions level "Developers" to allow
building suites *-proposed* and experimental:
_ this is low hanging fruit, requires a simple permissions test
function and another if statement. I can provide this by the
weekend
Adding support for other distributions:
Assumptions:
- distributions use gitlab for both permissions and projects code
to build
- distributions manage their own administrators for adding jenkins
jobs
- distributions use their own gitlab group
- distributions provide their own package repo amprolla setup etc
- we provide a separate releasebot tailored to their configuration
requirements.
Required work:
a patch to prepend the distro nickname to the handle
- this is easy, and I can deliver this sometime over the
weekend.
a patch to the templates or template logic
to define the mirrors to use instead of devuans
- this might also require a refactor of the
debian-jenkins-glue-buildenv package to ensure env-var
mirrors are respected.
- mirror definitions could be hard coded in the template as
environment variables - making the tempates distro specific.
**Push triggered builds:
We can use a webhook and a socket listener to trigger builds.
_Evilham has already begun work on implementing this.
Aims:
- uses existing releasebot - means all permissions are implemented
as
defined in gitlab.
- simple filter using gitlab group to determine whether to trigger
a build
* 2 possible options for triggering releasebot:
a) execute it directly.
_
pro - immediately starts build
con - webhook daemon runs where releasebot is located
- needs patching releasebot so it:
- copes with not having an associated issue to start with
- supports project and user arguments.
b) create an issue in the project and let releasebot start the
build as it does now.
_ pro - all the current notifications are sent as now.
- doesn't need to be run on the same server as releasebot
- good project records of who and when project builds
where executed.
_ con - up to 5 minute delay for the build to start.
I'd consider option (b) the quickest and most viable approach with
(a) a longer term plan and it's something Evilham can work on
independently.
Actions since last meeting:
- continued working on test suite for devuan-releasebot and pushed
to git
- shared lots of build infra knowledge and "bigger picture" ideas
with Evilham
Coming week:
- really busy with work (client Django project with deadlines
looming)
- will provide patches to releasebot for Developer to be able to
build *-proposed* & experimental suites.'
- will provide patches to releasebot to add distro/group prefix
to jenkins job name
## dimitsos
- I know that this might seem like a comic relief right now, and it's
really a long shot, but imagine Devuan running on that machine:
https://raptorcs.com/content/TL2WK2/intro.html
## fsmithred
- Made a live iso. ascii, openrc, eudev, openbox amd64.
http://distro.ibiblio.org/refracta/files/experimental/ascii_oblx_eudv_oprc-20170813_2222.iso
## golinux
- http://devuan.evilham.com/ci.png - this graphic needs to be on the
website. It used to be on the index page -
http://web.archive.org/web/20150720070928/https://devuan.org -
but got lost along the way. Was promised to be reinstated but that
never happened (or at least I haven't found it).
- Website needs updating. Commits are already in days/weeks ago.
- More updates pending but why bother if they just sit in limbo
- Posted to the forum http://defert.com/vincent/devuan-pour-tous/
another install guide in French.
## KatolaZ
Sorry, will not be there, due to another commitment.
- finished the setup of pkgmaster.devuan.org -- we can start to test
the mirror functionalities as soon as we have the signing key
- Still waiting for the signing key on amprolla-new.