Hi Steve, excellent feedback as always :)

On Wed, May 27, 2015 at 4:41 PM, Steve Litt <slitt@???>
wrote:

> On Wed, 27 May 2015 14:27:17 -0400
> Jude Nelson <judecn@???> wrote:
>
>
> > I've been thinking about the general form of this problem (and a
> > general solution) for months, and I think I'm almost ready to provide
> > a formal design document.
> >
> > [The Problem]
> > Many programs couple the business logic to the presentation logic.
> > The code that drives the UI is too intermingled with the code that
> > provides the application's core functionality to use them separately,
> > or develop on them independently.
>
> Hi Jude,
>
> Be careful what you wish for. MVC was created to solve this problem,
> and it looks to me like, in the hands of most developers, like MVC
> makes an unfathomable Easter Egg Hunt where a reasonably simple
> algorithm can't go six lines without the need to go look something up
> somewhere else. And yet, in the hands of most practitioners, my
> observation is they *still* manage to intersperse program logic, data
> and UI.
>
> Separation is good, but not all ideas for separation are good.
>

Yeah, MVC can get out of hand if used improperly. However, that's can be
because sometimes MVC isn't the right factoring in the first place, but
(for Web applications in particular) a lot of development frameworks force
developers to try to shoehorn their applications into it nevertheless.

I should point out, I don't see shui as an MVC-like framework. I see shui
as way to compose unrelated individual programs that each implement a
single UI element to create complex graphical applications, in the same
spirit as using the shell to compose unrelated individual programs that
each implement a data stream filter to create a pipeline that accomplishes
a complex data transformation. I'd use the shell with zenity to accomplish
what I'm proposing with shui, except that I additionally want to expose the
structure of the UI as a first-class aspect of the application as well.
This is because I want to let the user arrange the UI elements of an
application however they see fit without having to touch any code (i.e.
they can create and preserve whatever workflow they want irrespective of
the developer's intentions), and I want the user to be free to borrow UI
elements from one application and put them in others (i.e. we can never
lose features; only disable them).


>
> [clip]
>
> > [Motivation]
> > I was watching my SO create a (long) Powerpoint presentation the
> > other day, and she wanted to add a "fade in" animation to each bullet
> > point on each slide. Powerpoint (and every office suite I'm familiar
> > with) makes you do this by hand, bullet-point by bullet-point, with a
> > mouse. The UNIX hacker in me saw this, and wondered why it's not
> > possible to simply open a shell and run something like "for
> > ppt_bullet in $(ppt-ls "bullets"); do ppt-animate "fade-in"
> > "$ppt_bullet"; done". This would have saved an hour of tedious,
> > repetitive effort.
> >
> > The reason this isn't possible is because Powerpoint (and every other
> > office suite) does not offer a command-oriented UI. What could
> > Powerpoint have done? One option is to add a program-specific
> > command-language (like how vim and emacs do). However, I don't think
> > this the answer either, for two reasons:
> > (1) The UI (the command-language in this case) is still tightly
> > coupled to the program, and has rules and semantics that are only
> > applicable to the program.
>
> I think #1 above is a *good* thing. There's a reason it's called a
> *domain* specific language. It's the functions of this program's
> "business logic", and nothing more.
>
>
Yes, domain-specific languages have their place. However, all
Turing-complete languages are equivalently powerful. So, there had better
be a compelling reason to spend time designing a domain-specific language
that you then expect users to go learn just to use your program properly,
as opposed to embedding support for a more general language that the user
already knows (or can easily learn through 3rd party resources) and
shipping a library for that language that exposes your domain-specific
functionality.

It's a bit of digression, but the point I'm arguing is that unless the
programming paradigm itself is so specific to your problem domain that you
need a whole new programming language to describe its system of logical
axioms (this is rarely the case), I think developers and users are better
off with the latter option since it doesn't re-invent the wheel and doesn't
waste the user's time. For example, I'd do my best to design shui so that
it only ever needs to communicate with the subprocesses it runs via
environment variables, exit codes, signals, and pipes, thereby giving users
a wide selection of languages for programming applications.


>
> > (2) Without careful design, the user can't bring to bear existing
> > knowledge or existing tools to address problems. This is a general
> > gripe I have against vim and emacs too--if you're going to offer a
> > Turing-complete command language, why not offer one that is
> > widely-used already and lets people use external tools?
>
> That's just it. I think packaging a Turning-complete language is
> overkill, much the same as having the init system take charge of login
> logic. I'm an elder in the Church of the Parsimoneous Scope. I'd prefer
> to expose every business logic function, such that it's callable from
> the command prompt and other places. Those functions are the thin
> interface connecting business logic and appearance. There's nothing
> Turing complete about them. If you want Turing completedness, that's
> for the program
>

I think we're in agreement. With shui, each UI element should be a single,
distinct entity that you'd invoke and compose with others through a
higher-level command interpreter. Shui would be used in conjunction with a
set of UI elements akin to how the shell is used in conjunction with
coreutils. In the implementation, I'd make it so that each UI element ran
as separate subprocesses of shui, and communicated with the main shui
interpreter through the usual IPC mechanisms. UI elements would not
directly talk to each other; they could only share information through
files (thereby exposing the information necessary for the user to compose
other programs with shui, such as to control a shui program through the
shell or inject extra functionality into it).


>
> Years ago GoLUG's Gary Miller told me something I never forgot. When
> writing a program, he makes his entire user interface as input and
> output to a socket. That way, on his side of the socket, there's no
> need for Python-Tk, nor Curses, nor GTk nor tk nor HTML generation nor
> mod-apache-whatever: None of that complication. Nothing but business
> logic algorithms and the functions that implement them. Write it in
> simple, no-dependency C, Python, Perl, Ruby, Lua, whatever.
>
> Then Gary publicizes the commands that socket takes, so the other guy
> can write a program with Curses or Python-Tk or whatever, to present the
> info to the user, as needed. This presentation program has all the GUI
> molassas, but no business logic, so it's pretty straightforward. And
> you have complete separation of business logic from presentation,
> enforced by a socket.
>
> But of course, the thing on the far side of the socket can be more than
> a UI. It can be a script. It can be a program that turns a config file
> into a script. It can be a command prompt (using telnet) to tweak. You
> can accomplish almost anything, except adding a function or changing a
> function's actions on Gary's side of the program. If you want to do
> that, do it on Gary's side.
>
>
This sounds compelling, but (Gary, please correct me if I am wrong) it also
sounds like Gary is implicitly doing some application-level framing. That
is to say, it's not just a socket his software is using, it's a socket plus
a command language for sending structured requests and receiving structured
replies to and from the application. Moreover, it sounds like this command
language is specific to each application. In other words, Gary's approach
is equivalent to implementing a Web application--the UI and business logic
may talk through a socket, but the business logic dictates all the terms of
interaction to the client.

Gary, I would love to know more about your approach! Can you point me to
any public code repositories?

What makes shui's approach different is that it intentionally restricts its
command language to VFS operations, and it exposes state instead of RPC
endpoints. Everyone knows the VFS operations, so there's no new command
language you have to learn to access a shui-powered application's data.
Moreover, because shui applications expose state as files, they can make no
policy on how clients consume the data beyond controlling access to it
through permission bits and ownership. To draw a rough analogy, Gary's
approach is to RPC just as shui is to REST.


>
> >
> > What I would like the most is if Powerpoint were somehow designed so
> > that its runtime state was exposed to the system shell, so I could
> > operate on it directly. Powerpoint wouldn't need to ship with its
> > own custom language, and I could use whatever tools I wanted.
>
> I'm pretty sure Gary's method would do just that.
>

Qualified yes. If powerpoint exposed a method for interacting with
bullets, or exposed read and write operations that affected the bullets'
state, then Gary's method would work (i.e. whether or not you can do this
depends on whether or not there's a sequence of RPCs you can make to Gary's
powerpoint replacement that will enact the intended state changes). With
shui, as long as the required state is exposed as files, then I can come
along and apply my own state-changes myself, without shui-powerpoint
getting involved.

I personally think that for data-oriented applications like powerpoint,
shui's approach is more straight-forward, but that's just my opinion.


> [snip]
>
> > [Design Notes]
> > The idea I I'm working on can be thought of as a re-telling of the
> > SQUEAK story for the UNIX environment:
> > * The shell is the binding language between different aspects of the
> > application, as well as between the application and external tools
> > (this replaces Smalltalk).
> > * Persistent state is encoded as files
>
> As a djb enthusiast, I'm always happy when state is encoded as files
> and directories.
>
>
>
> > * Applications are encoded as directory hierarchies, where:
> > -- a directory represents either one aspect of the program (i.e. a
> > dialog box, a text area, a file explorer, etc.), or an aggregate of
> > multiple aspects.
> > -- leaf files are either persistent state for the aspect that their
> > parent directory represents, or executables that should be run in
> > response to an external event (i.e. from the user, from the passage
> > of time, or from another program).
>
> Be careful of those executables. Some fool will try to put business
> logic in the program that runs part of the UI.
>

Shui itself would perform the rendering, and would only run the UI element
code in order to carry out a desired action in response to some external
event (similar to traditional event-driven UI programming). It would be
linked against an existing widget toolkit to perform the actual rendering,
and (down the road) would define a way to dynamically load app-specific
widgets (which themselves would need to be programmable in the same way as
shui).

While we can't prevent it, we can take steps in the design to discourage
mixing UI and business logic. The desired effect would be to make it easy
to write stateless UI code, less easy to write stateful UI code that has
only public persistent state, but hard to write UI code that can have
private, hidden persistent state, and hard to couple together UI code in
different elements. For example, we might have shui unset the DISPLAY
environment variable, so UI elements can't easily launch their own UIs. We
might also intentionally do nothing to help UI elements communicate with
each other, except through writing files to disk (we might even go so far
as to use seccomp or capsicum to enforce this), so they can't prevent
composition with other programs. In addition, we (at a distro level) would
only include extra widget implementations that didn't have this antipattern.


> >
> > For example, a simple network manager might look like:
> >
> > $ cd network-manager/ && find .
> > .
> > ./window-main
> > ./window-main/panel-buttons
> > ./window-main/panel-buttons/button-Cancel
> > ./window-main/panel-buttons/button-Cancel/data
> > ./window-main/panel-buttons/button-Cancel/on_click.sh
> > ./window-main/panel-buttons/button-Connect
> > ./window-main/panel-buttons/button-Connect/data
> > ./window-main/panel-buttons/button-Connect/on_click.sh
> > ./window-main/selectbrowser-networks
> > ./window-main/selectbrowser-networks/on_timer.sh
> > ./window-main/selectbrowser-networks/data
> > ./window-main/selectbrowser-networks/on_select.sh
> > ./window-main/menubar-main
> > ./window-main/menubar-main/help
> > ./window-main/menubar-main/help/002-Credits
> > ./window-main/menubar-main/help/001-About
> > ./window-main/menubar-main/file
> > ./window-main/menubar-main/file/001-Quit
>
> I'm hoping that the preceding does nothing, nothing, NOTHING except ask
> questions of the business logic module, and display the answers.
>
> Exactly what are the .sh files and what do they do? Do they simply call
> functions from the business logic module (good), or are they intended
> for main-window to add its own business logic (bad), or incorporate code
> from the business logic module into the .sh file (probably bad)?
>

The business logic in this particular example are the network and wireless
command-line tools. Shui and this application are a thin wrapper around
them, and knows what to render based on the names and structures of the
directories. The set of .sh files implement state transitions in the
application by calling out to the business logic when appropriate. They
would otherwise behave like event handlers in conventional UI programming.
Shui would run them as subprocesses in response to UI interaction, and they
would do the following:
* selectbrowser-networks/on_timer.sh runs "iwlist wlan0 scan" or the like
every so often, and writes the useful network information to e.g.
~/.shui/network-manager/last_scan.txt (or some well-known location).
* selectbrowser-networks/on_select.sh writes the information about the
selected network to e.g. ~/.shui/network-manager/selected_network.txt (or
some well-known location).
* button-Connect/on_click.sh queries the selected network (written to
e.g.~/.shui/network-manager/selected_network.txt) and runs the right
sequence of ip, wpa_supplicant, iw, etc. to connect to it. On success, it
might write information about the network to
~/.shui/network-manager/connected_network.txt.
* button-Cancel/on_click.sh asks the shui parent process to exit, since the
user is done with the program.
* menubar-main/file/001-Quit asks the shui parent process to exit as well.

Other UI elements and other programs that want to know what network the
host is connected to might check
~/.shui/network-manager/connected_network.txt to find out. The exact path
is an implementation detail, but as long as it's a well-known canonical
location, it should be fine (and if you don't like the location, you can
symlink it to the location you want). For example, a notification daemon
might monitor ~/.shui/network-manager/connected_network.txt and display the
access point's ESSID as a notification when you successfully changed the
connection.

main-window is merely an aggregation of its immediate children UI
elements. It knows nothing about what they do--it only tells shui to
render them into a single window called "main."

menubar-main is a menu, with "help" and "file" submenus. help/001-About
and help/002-Credits are text data to be displayed in a simple dialog.
file/001-Quit would be a script that just signaled shui to exit.


> If you're going to do this, have you given any thought to making a
> Rapid Application Development (RAD) program out of it? You appear to
> already be doing window generation. Add in some templating, and who
> knows how fast development might get.


I was considering using shui to create a shui-powered shui-app-builder at
some point :)

My goal at the end of the day with shui is to be able to build and maintain
with minimal effort a set of applications that I can interact with either
graphically or programmatically. I'm exceedingly tired of this "everything
has to be point-and-click and users should never use the command-line"
design philosophy that seems to pervade the desktop world, and I see shui
as one way to fix that. I'd go so far as to make it possible to create
panels, root windows, and system configuration tools, so I could combine
shui with a window manager and have a fully-functional desktop environment
where I can have a consistent interface for extending and scripting the
behavior of each program with minimal effort.


>
> Or how bad. I have no doubt that RAILS is fast to develop, but I
> shudder every time I look at RAILS "code".


I'd be curious to know what kinds of antipatterns you've discovered, so we
can design shui to discourage them.

-Jude