Devuan Notes for Aug 16(17) 2017

Pad:
https://pad.dyne.org/code/#/1/edit/nVuY1HnoWtgH6bmSyrWKxQ/tBEct7e9LKxhIY4Rx7I7dfJN

Present: Centurion, Evilham, fsmithred, rrq, golinux, dimitsos (pad,
irc), NewGnuGuy (observing)

Missing: The usual suspects.

## Infrastructure discussion (priority)

-
https://pad.dyne.org/code/#/1/edit/usQoFADFDsJ9UAFbVm0H7A/FPGyscaoxWTAqqgscT1bw7VA

## Build improvements discussion

-
https://pad.dyne.org/code/#/1/edit/nXiySd0FPHvG8pBgGgImxA/aJ87TdwIEWiIk96GZcFwypnb

## Centurion Dan
It feels to me like we have a number of key issues in play:

1) How to allow other distributions to use our CI system without
compromising on security? To me there are several factors to consider:

      a) What benefit does hosting each of these other distributions
bring to Devuan and does that benefit offset the cost of resources
required to provide the systems including developer/admin/training time?

      b) Do we allow them to cherry-pick our resources - ie use jenkins
without requiring them to use git.devuan.org (because gdo currently
provides permissions to authorise actions on jenkins)?

      c) Will their use of our CI system constrain our ability to further
develop/enhance/augment our CI system?

      d) How do we arbitrate potentially scarce resources (ie buildhosts)
when there is competition for those resources (think when urgent
security updates need to be built and another project has queued up a
massive number of jobs?  (_Currently we don't but jenkins does have the
capability, to set constraints on a per user/group basis for number of
running builds etc but some time would be required to consider and set
this up)?

   2) How to make triggering builds in the CI system easier then it is
     now?

3) How to provide finer grained (ie per suite) build permissions?

   4) Is there a real security vulnerability with gitlab that makes it a
     now unwise to continue to use as the federated authentication
     provider for our git and CI, and Devuan as a whole system?

**With respect to the scorsh proposal:

I'll agree that scorsh as designed provides a solution to some of
these stated problems, but I believe it does so at the cost of
transparancy, and adds significant complexity and ignores the bigger
picture beyond just building packages

**My concerns about scorsh based on it's DPI/RFC:**

- Scorsh has it's own user database that is not representative of the
existing user database in gitlab. This is problematic as users will
need to be administered in 2 places.
** A possible solution is that scorsh provides an OAuth integrated
website to allow users to manage their own PGP/GPG keys. **

   - Scorsh adds a separate permissions system for authorisation - one
which is entirely disconnected and doesn't respect any of the existing
gitlab federated permissions.  This means we will have two separate sets
of permissions to authorise:
   a) who can upload to git.d.o on a project and branch level
   b) who can build the package of a project for a suite
   _ To me there is no reason to differentiate who can commit to
     suites/jessie and who can build suites/jessie.  If we don't trust
     the committers commits to be built they shouldn't be directly
    committed to that branch._

   ** A solution to this will be to make scorsh use gitlabs permissions
     (available via the api) to ensure anybody that can push to a branch
    in gitlab also can trigger the build of that package.  This obviates
    implementing the user self management of their own keys as suggested
    above. **

   - The Scorsh design requires a refactored version of releasebot that:
   i) bypasses releasebots usual permissions model - because scorsh
     implements it's own
   ii) potentially requires different templates for other distributions
     and thus requires adding code to select those different templates.
      __+ this probably means a fork of releasebot is required, resulting
    in both duplication of function and risk that templates diverge.
    Thus a job add or modification in scorsh results in a different
    jenkins job config to that which is provided by gitlab triggered
    action.__

**Thoughts on Refactoring Releasebot to solve these problems:

   Adding additional build permissions level "Developers" to allow
   building suites *-proposed* and experimental:
   _ this is low hanging fruit, requires a simple permissions test
     function and another if statement.  I can provide this by the 
weekend

   Adding support for other distributions:
     Assumptions:
      - distributions use gitlab for both permissions and projects code
        to build
      - distributions manage their own administrators for adding jenkins
        jobs
      - distributions use their own gitlab group
      - distributions provide their own package repo amprolla setup etc
      - we provide a separate releasebot tailored to their configuration
        requirements.

      Required work:
         a patch to prepend the distro nickname to the handle
           - this is easy, and I can deliver this sometime over the
            weekend.

         a patch to the templates or template logic
           to define the mirrors to use instead of devuans
           - this might also require a refactor of the
             debian-jenkins-glue-buildenv package to ensure env-var
            mirrors are respected.
           - mirror definitions could be hard coded in the template as
             environment variables - making the tempates distro specific.

**Push triggered builds:

     We can use a webhook and a socket listener to trigger builds.
     _Evilham has already begun work on implementing this.

     Aims:
     - uses existing releasebot - means all permissions are implemented
      as
       defined in gitlab.
     - simple filter using gitlab group to determine whether to trigger
       a build
     * 2 possible options for triggering releasebot:
       a) execute it directly.
       _
          pro - immediately starts build
          con - webhook daemon runs where releasebot is located
              - needs patching releasebot so it:
              - copes with not having an associated issue to start with
              - supports project and user arguments.

       b) create an issue in the project and let releasebot start the
         build as it does now.
          _ pro - all the current notifications are sent as now.
                - doesn't need to be run on the same server as releasebot
                - good project records of who and when project builds
                 where executed.
          _ con - up to 5 minute delay for the build to start.

       I'd consider option (b) the quickest and most viable approach with
      (a) a longer term plan and it's something Evilham can work on
       independently.

   Actions since last meeting:
     - continued working on test suite for devuan-releasebot and pushed
       to git
     - shared lots of build infra knowledge and "bigger picture" ideas
      with Evilham

   Coming week:
     - really busy with work (client Django project with deadlines
      looming)
     - will provide patches to releasebot for Developer to be able to
       build *-proposed* & experimental suites.'
     - will provide patches to releasebot to add distro/group prefix
       to jenkins job name

## dimitsos
   - I know that this might seem like a comic relief right now, and it's
     really a long shot, but imagine Devuan running on that machine:
     https://raptorcs.com/content/TL2WK2/intro.html

## fsmithred
- Made a live iso. ascii, openrc, eudev, openbox amd64.

http://distro.ibiblio.org/refracta/files/experimental/ascii_oblx_eudv_oprc-20170813_2222.iso

## golinux
   - http://devuan.evilham.com/ci.png - this graphic needs to be on the
    website.  It used to be on the index page -
    http://web.archive.org/web/20150720070928/https://devuan.org -
    but got lost along the way.  Was promised to be reinstated but that
    never happened (or at least I haven't found it).
   - Website needs updating.  Commits are already in days/weeks ago.
   - More updates pending but why bother if they just sit in limbo
   - Posted to the forum http://defert.com/vincent/devuan-pour-tous/
    another install guide in French.

## KatolaZ
    Sorry, will not be there, due to another commitment.

    - finished the setup of pkgmaster.devuan.org -- we can start to test
     the mirror functionalities as soon as we have the signing key
    - Still waiting for the signing key on amprolla-new.