On Thu, Nov 19, 2020 at 04:36:00PM -0500, Mason Loring Bliss wrote:
> On Thu, Nov 19, 2020 at 11:16:11AM +0100, Ivan J. wrote:
>
> > Also "with a configurable timeout" sounds wrong. Instead, I would
> > implement proper error handling and cleanup upon error.
>
> That's one of the things noted about Amprolla - it wasn't noticing
> connections hanging. The only way to address this is to assert a timeout.

I misunderstood then. I thought the timeout was somehow related to
lockfiles.

> That said, the basic plan for lurc is to have many small parts. As such a
> timeout or error will be a failure of a subpart, and dependent parts will
> understand that a depenendency has failed. The error will be logged, and
> monitoring should notice errors and alert folks who can have a look. They
> can either re-run the individual piece that broke or await the next overall
> run.
>
> That said, the notion of clean-up is good, and I'll test download failures
> mid-flight. I can download into a staging error and only move files into
> the working area if they pass muster.
>
>
> > I recommend parsing Release files rather than trusting HTTP headers,
>
> We want to trust HTTP headers for automated download, or we'll be
> downloading large files constantly, which is far too heavy. The --force
> flag will be a way around this should the stars converge such that a new
> upstream file somehow doesn't have a changed modification header, but this
> should be quite rare.

Release files are not large at all. Also, if you are getting HTTP
headers, you're getting the Release file as well ;)

Best regards,
Ivan