On 2025-10-23 10:07, onefang wrote:
> On 2025-10-23 09:55:03, sdomi wrote:
>> On 2025-10-23 08:25, onefang wrote:
>> > On 2025-10-23 05:59:51, Shengqi Chen wrote:
>> > > Hi,
>> > >
>> > > > -----Original Message-----
>> > > > From: devuan-mirrors <devuan-mirrors-bounces@???> On Behalf Of
>> > > > onefang
>> > > > Sent: Thursday, October 23, 2025 1:11 PM
>> > > >
>> > > > G'day, Devuan's package mirror herder here.
>> > > >
>> > > > I've been working on a new update script for Devuan package mirrors.
>> > > >
>> > > > https://sledjhamr.org/cgit/devuan_package_mirror_sync/
>> > > >
>> > > > Which tries to update a Devuan package mirror with minimal issues. It
>> > > > should help prevent "this mirror is currently updating and might give
>> > > > errors", or at least shrink that window down to fractions of a second.
>> > >
>> > > It looks rather like Debian's two-stage rsync used in ftpsync.
>> >
>> > I've looked at ftpsync, it's much more complicated than what I think we
>> > need, but yes a two stage rsync.
>>
>> Seconding that. Debian's convoluted mirrorscript (and their insistence
>> that
>> everyone needs to use it) is one of the top reasons why I don't run a
>> Debian
>> mirror alongside Devuan. As long as something boils down to one (...
>> two?)
>> rsync calls, I don't buy that we need to migrate to a "real"
>> programming
>> language. Let's not over-engineer :)
>>
>> > > And we (with my TUNA mirror's admin hat on) have also implemented
>> > > our own version.
>> >
>> > Go isn't one of my 100 programming languages, and I still haven't
>> > decided
>> > what language 101 will be.
>> >
>> > > I suppose devuan does not have much deviation with Debian's folder
>> > > structure,
>> > > so we could still re-use those mechanisms safely?
>> >
>> > We could, but I'm trying to simplify and speed up things.
>> >
>> > > Thanks,
>> > > Shengqi Chen
>>
>> As likely every other mirror operator, I have my own bespoke script
>> for the
>> purpose - which did already include --delay-updates, and
>> --delete-after. Or
>> is the new script different in some other way?
>>
>> As an aside: I don't like how we're calling rsync twice, and I don't
>> fully
>> understand the intent (especially with --delete-delay only being in
>> one of
>> those calls). Care to elaborate about the source of the problem?
>> Because to
>> me, this sounds like core mirror doesn't get updated atomically, and
>> this is
>> a workaround? I'm confused.
>
> The main problem is symlinks. The various metadata files are sometimes
> symlinks, but rsync refuses to create the file the link points to
> before
> it creates the link. So the first one excludes the metadata files, and
> the delete is in the second one to delete things when everything is
> settled.
>
> I tried to reduce it to a single rsync, but it just wasn't gonna
> happen.
> If anyone can do that, let us know.

Oh! Now this is behavior that i haven't encountered before. Would need
to make an entire repro setup to debug this. Not sure if I have the
time, but this feels like we're either misusing some rsync switch, or
hitting a bug.

Will get back to you if I find anything.

~sdomi