:: Re: [DNG] kernel drivers [WAS: How …
Top Pagina
Delete this message
Reply to this message
Auteur: Didier Kryn
Datum:  
Aan: dng
Onderwerp: Re: [DNG] kernel drivers [WAS: How long should I expect to wait for openrc to be ready in devuan ascii]
Le 11/07/2017 à 14:30, Enrico Weigelt, metux IT consult a écrit :
> On 05.07.2017 23:30, Didier Kryn wrote:
>
> > And there are a whole menagerie of addressing and transfer
> protocols, > including chained dma transfers
>
> Especially things like DMA are naturally kernel domain. I wouldn't ever
> allow any direct access to dma controllers from userland.
>


     The system isn't a server open to the public. It's a kind of 
industrial process (though it's for fundamental research). Only a 
limited number of users have an account on the hosts and only members of 
the vme group are given permission to access the devices associated to 
VME. In production, only one user runs a few applications (essentially 
one), some automatically on reboots and some under remote control. A 
whole library written in Ada comes between the applications and the 
driver; it adds a sufficient level of abstraction and a few facilities 
and takes care of safety controls. The applications are all written in 
Ada, of course. We don't fear malevolent actions, and, IMHO, safety is 
better granted by an Ada library than by any driver written in C.


> > In our case it is waveform digitizers and the interaction is so >
> complex that it cannot be done in a kernel module
>
> What exactly is so complex about that ? Smells like a typical task
> for IIO to me.


     The waveform digitizers boards have 8 channels of 8-bit 500MSPS 
flash-ADCs, each with 1024 buffers of 4096 bytes (samples). The 
(external) trigger comes after the data has been recorded; the many 
buffers grant asynchronism between data taking and read-out. The VME 
master can skip buffers (events) or read them, totally or partially. The 
VME master (the single-board computer) monitors the status of buffers 
and must free them before the digitizers can overwrite them. Before data 
transfer, the range of samples from the selected channels are arranged 
into a special buffer of the WFD boards, following detailed instructions 
from the VME master, before they are transferred using the 2eSST320 
protocol (currently the fastest VME64x protocol). The amount of data 
transferred for each buffer depends on external conditions comming from 
a 16-bit binary input present on each board and part of meta-data 
associated to each trigger; the VME master decides what to do based on 
metadata assembled from 4 WFD boards. These metadata are generated by a 
set of very fast measurements associated to the triggering system 
(another subsystem of our set-up).

>
>> Some others consider there should be one driver per VME device you
> > want to talk to, on top of the raw VME driver.
>
> Exactly.
>
>> These people only ever had to deal with simplistic devices. Gabriel
>> Paubert has never seen an application transfering more than a few
>> bytes, while we are transferring data at 320MB/s.
>
> Those kind of things usually involve proper timing, caring about
> caching, etc, etc. Natural kernel domain.


     It's a little more complicated, as explained above :-)

>
>>> Which absurdities, exactly ?
>>      Bus errors were not reported to the application but instead 
>> caused a message to syslog! This makes it hard to use; or your 
>> application should monitor the log files!

>
> Yes, that's hackish, of course.
>
>>      This isn't the case here. AFAIU, their BSP contains a VME driver 
>> and a device tree. I think it's all GPL. Their buzyness is to send 
>> hardware and they make sure their clients can use it.

>
> Looks like your vendor is an exception.
>
>>      These are SBCs with everything soldered: VME, flash memory, 
>> UARTs, USB hub, Sata controllers, Ethernet ports etc... And you just 
>> plug it into a VME crate. It makes no sense to develop a board like 
>> this to use a dozen of them. We made the digitizers and it was 
>> already a big deal for our small team.

>>
>>      Could you develop what is IIO for DAQ devices? 

>
> What's the difference between you digitizes and usual DAQ devices ?
> What do you actually do with them ?
>
> Some clear requirements on the table, please ;-)


     We have two detectors running. On each of them, a sophisticated 
trigger logic decides when interesting data has been seen and triggers 
400 channels of waveform digitizers (like 400 synchronous oscilloscope 
channels). 64 bit of trigger-related meta-data are transmitted to the 
read-out subsystem by the trigger system via the waveform digitizers 
digital input. There are a few hundreds triggers per second. The data 
are collected from 5 VME masters (in 5 VME64x crates) through 1gb/s 
ethernet by a Dell Poweredge server, re-arranged, compressed and written 
to disk before further transmission to a big computer center. 
Compression must be done before writing to avoid saturate the disk 
bandwidth.


>
>> Is it something allowing to write drivers in user space?
>
> What kind of drivers do you wanna have in userland ? And why ?
>


     I think what I would do now would be to mmap() the VME bridge from 
userspace right away (I've read it's possible with /dev/mem). This way 
it would work with any kernel release and the API wouldn't change. But 
it's too late now, the project will finish by the end of the year and 
I'm retired.


     Sorry, this thread is pretty off-topic, but it reminds that Linux 
isn't only used on web servers :-)


     Didier