Fred:
...
> I need to work with a nonstandard serial three button mouse without
> scroll wheel. It needs a baud rate specification and gpm has an
> option for this.
...
> I know that gpm works with this mouse.

Since gpm works with the mouse I wouldn't say it is a "non-standard"
one. What mouse protocol does the mouse speak ?

Common protocols are mouseman and msmouse.
You can see what protocols are supported by gpm by doing:

gpm -m /dev/null -t help

Unfortunately you have to do that as root, it doesn't matter what
you write as the device, it can be any string.

You can also find out about mouse protocols with

man mouse

Examples of mouse protocols:

https://web.stanford.edu/class/ee281/projects/aut2002/yingzong-mouse/media/Serial%20Mouse%20Detection.pdf
https://www.kryslix.com/nsfaq/Q.12.html

Example mouse code (if you want to build a mouse yourself):

https://aspodata.se/git/openhw/boards_arm_aspo/mouse/mouse.c

////////
> What program is used as mouse driver on Beowulf and Chimaera?

In a text mode terminal you can use gpm or something similar.

In wayland, I don't know, it seems libinput is used by wayland.

In X11 you could use the mouse driver:

https://www.x.org/releases/current/doc/man/man4/mousedrv.4.xhtml
https://packages.debian.org/bullseye/xserver-xorg-input-mouse

You have to configure the mouse driver yourself, udev is of no help
for a serial mouse. You can have multiple serial mice connected and
used. Example xorg.conf extracts:

Section "InputDevice"
        Identifier      "Configured Mouse"
        Driver          "mouse"
        Option          "CorePointer"
        Option          "Device"                "/dev/ttyS4"
        Option          "Protocol"              "MouseMan"
EndSection

Section "InputDevice"
        Identifier      "Configured MSMouse"
        Driver          "mouse"
        Option          "SendCoreEvents"
        Option          "Device"                "/dev/ttyS5"
        Option          "Protocol"              "Microsoft"
EndSection

Todays usb mice can use:

Section "InputDevice"
        Identifier      "Generic Mouse"
        Driver          "mouse"
        Option          "SendCoreEvents"        "true"
        Option          "Device"                "/dev/input/mice"
        Option          "Protocol"              "auto"
        Option          "ZAxisMapping"          "4 5"
EndSection

It has apperantly been removed in favour of libinput upstream:

https://www.gentoo.org/support/news-items/2020-04-03-deprecation-of-legacy-x11-input-drivers.html

Regards,
/Karl Hammar