On Fri, 19 Apr 2019, Daniel Reurich wrote:

> Hi,
>
> ci.devuan.org - our jenkins server is currently down. This is due to a
> reboot failure after a kernel update that I installed.

this intervention was not planned not communicated; it also was on a
old infrastructure to which we have no stable reach, because is
maintained by nextime and therefore needing extra coordination
measures to insure interventions.

I am unconfortable knowing that anyone of the caretaker can act
unilaterally on such issues, raising risks of emergency interventions
which then affect everyone schedule.

we do need to coordinate on these tasks and find periods in which
everyone affected / responsible for the infrastructure bit is
available.

I went a long way yesterday urging nextime to help, he is just packing
today for a trip offline for the coming two weeks and the situation is
very uncomfortable as works were schedule and still pending also for
the DNS administration access. He will do his best today to fix that
so we can rotate the DNS on a new machine.

after that, we should take the occasion to rebuild the CI with better
criteria, since the old setup was suboptimal. at dyne we (well, mostly
parazyd) already setup two more building farms CIs (one for DECODE and
one for maemo-leste) and have fixed a number of issues. Therefore I
kindly ask parazyd and ralph and evilham for their availability
setting up a new CI machine on the ganeti network, where parazyd can
install and plan a new jenkins instance, which I understand won't cost
him too much time since he has a well documented and replicable
procedure for that now.

meanwhile we can simply consider the CI unavailable for the period of
Easter, which I hope you all manage to enjoy. we needed to fix this
bit anyway so lets be constructive and do it without letting rush take
over quality.

ciao