Interfaces naturally emerge as software gets broken down into parts communicating with one another. The larger and more deliberate structures emerge from a deliberate attempt to organize the development process itself. [fn:Liskov2008] Structure often emerge directly from division of labor: as teams take on independent tasks, interfaces are established betweeen domains they become responsible for. (Conway’s Law)

Software developers are responsible for systems built out of very small atoms while ultimately performing tasks for their users of a much greater magnitude. Dijkstra showed this by computing the ratio between grains of time at the lowest and largest atoms of the system (from say, CPU instructions to a human interaction with the system) The span was already quite large by Dijkstra’s time, of about 10^9. Today this ratio would be at least above 10^12 (see grain ratios)

This large span has to be managed somehow, often through hierarchies of layers. [fn:EWD361]

[fn:Liskov2008] https://youtu.be/O6By99JW_V8?t=1647 On the desire to partition large systems into modules with intentional interfaces in order to manage the complexity of the software development at scale during the creation of the Venus operating system.

[fn:EWD361] https://www.cs.utexas.edu/~EWD/transcriptions/EWD03xx/EWD361.html

For besides the need of precision and explicitness, the programmer is faced with a problem of size that seems unique to the programmer profession. When dealing with “mastered complexity”, the idea of a hierarchy seems to be a key concept. But the notion of a hierarchy implies that what at one level is regarded as an unanalyzed unit, is regarded as a composite object at the next lower lever of greater detail, for which the appropriate grain (say, of time or space) is an order of magnitude smaller than the corresponding grain appropriate at the next higher level. As a result the number of levels that can meaningfully be distinguished in a hierarchical composition is kind of proportional to the logarithm of the ratio between the largest and the smallest grain. In programming, where the total computation may take an hour, while the smallest time grain is in the order of a microsecond, we have an environment in which this ratio can easily exceed 10^9 and I know of no other environment in which a single technology has to encompass so wide a span.

I am looking at practicionners not theoretical material.

https://soundcloud.com/podcastcode/6-dont-make-me-write-ui

https://vimeo.com/10556923 at around 01:02 talking about interfaces

API.md

Cited as a good API by Casey Muratori

http://akkartik.name/about

Cautionary tale about making interfaces internal to a system too rigid, which results in them not being improved. It is an angle where the context matters a lot. What is the audience of a piece and its interface? What communication delays are there?

On one end we have systems like the Linux kernel, which try to preserve external as well as internal interfaces stable. On the other end we have software whose internal structures shouldn’t be solidified too early.

Especially since some parts of the break-down that generates interfaces is largely accidental and a result of “problem solving by divide and conquer”

https://www.youtube.com/watch?v=RT46MpK39rQ&feature=youtu.be&t=27m51s

http://www.nrl.navy.mil/itd/chacs/sites/www.nrl.navy.mil.itd.chacs/files/pdfs/Heitmeyer2002.pdf http://web.stanford.edu/class/cs99r/readings/parnas1.pdf

http://www.pcg-random.org/posts/ease-of-use-without-loss-of-power.html http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0347r0.html

http://ozlabs.org/~rusty/index.cgi/tech/2008-04-01.html

http://antirez.com/news/107

https://anteru.net/2016/05/01/3249/

http://www4.in.tum.de/~blanchet/api-design.pdf

http://twitter.com/mbostock/status/681561150127878144

It’s funny how writing documentation can spur redesign: it’s easier to simplify a complex API than try to document it completely”

What Makes Software Good

If you can’t come up with a good name for a method or a class, your design might be flawed

09-19-15 | Library Writing Realizations http://www.cbloom.com/rants.html

Hard to use those from the Efficient Programming with Components

Lecture 7 part 2

At that time Stepanov reveals that actual usage is necessary to write the generic operation:

https://youtu.be/S2iTfUyVOcY?list=PLHxtyCq_WDLXryyw91lahwdtpZsmo4BGD&t=273

Most of the times writing a program is an exploratory walk.

Lecture 11 Part 1, where he briefly mentions iterators and boost ranges:

https://youtu.be/84gHZgPCf1s?list=PLHxtyCq_WDLXryyw91lahwdtpZsmo4BGD&t=1455

Good choice of operator https://youtu.be/2mU8CTO2vSc?list=PLHxtyCq_WDLXryyw91lahwdtpZsmo4BGD&t=2922

And the reasoning behind advance and distance vs +=

https://youtu.be/_8hN232WNYU?list=PLHxtyCq_WDLXryyw91lahwdtpZsmo4BGD&t=276

You cannot know if you’re right until you found thing in context.

https://youtu.be/Dly8Ff4aDp8?list=PLHxtyCq_WDLXryyw91lahwdtpZsmo4BGD&t=1797

Write usage code first

The interface of rotate is designed to make it easy to compose. Its return value.

Other principle: don’t throw away information that your algorithm has already computed.

https://gist.github.com/pervognsen/d57cdc165e79a21637fe5a721375afba https://gist.github.com/vurtun/192cac1f1818417d7b4067d60e4fe921

• Write Usage Code First
• Don’t Throw Away Information that was computed
• Iterate

An algorithm should first be written in context i.e. the usage code is written first and the algorithm extracted from it.

It is even better if you have experimented with multiple usages so as to discover the best interface for the algorithm. Don’t worry however and be ready to revisit your API multiple times, as it is impossible to get right on the first try.

Anything that your algorithm is computing as part of its operation is of potential use by the user of this algorithm: don’t throw it away!

An example is a find function. It could return whether it found an element or not (bool) However in the process of finding that element it must have known the position of that element. So better return the position and not only a boolean. The position will be of great use for the user of your algorithm.

This is more about exploratory design than anything else.

Simon Cozens in a FOSDEM2015 talk about the SILE typesetting system:

https://youtu.be/5BIP_N9qQm4?t=1282

When you are implementign a subsystem, always implement it more than once. (…) that tells you where the separation of concerns lies.

One of the best piece of programming advice I ever got was from a guy I worked called Tony Bogen. He said that when implementing a subsystem always implement it more than once. And have two or three different variations of that because that tells you where the separation of concerns lies. And that’s been very helpful. Everytime I don’t do this I come to regret this because I generally end up writing that subsystem multiple times anyway.

See my “Commentary And Notes” notebook.

Reference: Granny Aninmation System 2.0

A common goal is to achieve code reuse.

Introduces the notion of Integration Discontinuity. This is basically what happens when the relationship between the user and the component being used starts growing to a point where the reused component does not provide what’s necessary for its user.

Theory and practice?

A component promises something but how is it in practice. Granny, first version built on reasonable code principles. Lots of people had problems integrating it. What happened?

The second incarnation was built after we learnt about it and this is our assesment

There are multiple tiers as you start integrating an API. They correspond to multiple stages of integration.

These tiers call for various characteristics. The idea though is that you let users of your API move between phases of integration so that they keep options open and don’t suffer from a discontinuity as they encounter a block.

http://wiki.qt.io/API-Design-Principles

https://gist.github.com/uucidl/68d471b05c3a82d0f0556274f57cf6a3 http://ariya.ofilabs.com/2011/08/hall-of-api-shame-boolean-trap.html

tl;dr:

A good API is optimized for reading code not writing. One easy to spot API mistake is the “Boolean Trap.” It can be summarized with the following rule: “It is almost invariably a mistake to add a bool parameter to an existing function.

Also https://blog.ometer.com/2011/01/20/boolean-parameters-are-wrong/

TODO: enumerate the effects of adding a bool to an API entry point, depending on various properties of that entry point and the generated effect. (With respect to granularity/orthogonality/stability of interface etc..)

I’m now super careful with adding bool parameters to the public imgui api. Many bools became an api mess/issue in the long run.

So I have a public function and want to add a bool parameter to it. Other solutions seem overkill for now (Two entry points? Flags? Enum?)

https://www.youtube.com/watch?v=heh4OeB9A-c

http://research.microsoft.com/en-us/um/people/blampson/70-SoftwareComponents/70-SoftwareComponents.htm an answer by stepanov: http://www.stepanovpapers.com/Industrializing%20Software%20Development.ppt https://softwareengineering.stackexchange.com/questions/221615/why-do-dynamic-languages-make-it-more-difficult-to-maintain-large-codebases/221658#221658

on that topic: http://www.uh.edu/engines/epi1252.htm http://st.inf.tu-dresden.de/files/teaching/ss10/cbse/01-introduction.pdf http://www.cl.cam.ac.uk/~srk31/research/talks/kell16operating-slides.pdf

http://research.microsoft.com/en-us/um/people/blampson/33-Hints/WebPage.html

http://repository.cmu.edu/cgi/viewcontent.cgi?article=2979&context=compsci https://www.cs.umd.edu/class/spring2003/cmsc838p/Design/criteria.pdf

“The people who rely on the compat layers don’t care enough to maintain it. The people who work on the mainline system don’t care about the compat layers because they don’t use them. The cultures aren’t aligned in the same direction. Compat layers rot very quickly . ” – Theo De Raadt

aka the REST API paper

“The strong typing of object-oriented languages encourages narrowly defined packages that are hard to reuse. Each package requires objects of a specific type; if two packages are to work together, conversion code must be written to translate between the types required by the packages.” [ John K. Ousterhout]

Also notes about reuse’s failure.

http://loup-vaillant.fr/articles/implemented-my-own-crypto

http://staltz.com/api-design-tips-for-libraries.html

http://timperrett.com/2016/11/12/frameworks-are-fundimentally-broken/

https://www.sebastiansylvan.com/post/matrix_naming_convention/

@url: https://flutter.io/design-principles/#introduction @title: Flutter Design Principles

Breaking API changes are treated as specific items worthy of attention. Proposals are written and summarized.

• Justification is made
• A migration path from old to new code is described
• Contact for supporting people to move old to new API

Weighting of API stability versus benefits will determine whether the breaking change is to be made.

Transition period by way of annotating old code as deprecated is to be introduced. (using tags such as @escape{@deprecated(‘Description’)})

1. Avoid hard to maintain data-retention/duplication

@quote{There should be no objects that represent live state that reflects some other state, since they are expensive to maintain. e.g. no HTMLCollection}

I.e. they discourage data-retention, as it requires synchronization between layers. HTMLCollection

1. Easy access implies cheap access

If something is implemented via what looks syntactically cheap, it should also be cheap.

1. Expensive operations should not be exposed as synchronous procedure calls
2. APIs should be arranged in “physical” levels.

• Convenience APIs layered on top of lower-level APIs
• Scope of each level is made as narrow as possible

1. Unsafe constructs are not promoted as regular APIs

ex: low-level construction of executable code from unsafe/user input pieces

1. Adapter APIs should be complete

When wrapping another API, faithfully wrap the complete API so as to minimize surprises and avoid creating integration surprises.

@url: https://code.kiwi.com/code-design-principles-for-public-apis-of-modules-6a43aaf26624 @title: Code Design Principles for Public APIs of Modules

Main actions on a module:

• Creation from scratch
• Redesign

Heuristics

Names

Remove anything that appears unnecessary (information compression) .. without going as far as hurting redability.

Example of a name that can be shortened, kw.booking.additional_booking_management could be better written kw.booking.additional_booking, as “management” does not appear to contribute much meaning.

Goals for better names:

• legible useage code, emphasizing its own logic (names that are too long might disrupt legibility of the useage code)
• names that are easy to be kept consistent (i.e. don’t invite variations)

Insights for better APIs

• how many engineers would maintain that module
• how many engineers would commonly write code using that module
• how frequently code using this API might be read over time (times/day)
• how long might this module be used
• consider what the user will see

This should help you decide the level of quality this API and implementation should be at.

Before making an implementation clean, think about the user first, and make the API clean. Write tests, prototype useage code.

Procedures

• keep number of positional parameters small (< 3)
• procedures are designed to work on data (if the procedure of a verb, its object), so design/specify that

<@pkhuong> Oct 9 Reading Fluent Python… does the official Python documentation really recommend hashing objects by combining their fields’ hashes with xor?

<@pervognsen> Oct 9 Huh? I’ve always just hashed a tuple of the fields…

I’ve definitely seen recommendations like that in various Java books, sadly, but Python makes it so easy to just hash a tuple.

Incidentally, one of my favorite general tricks is to exploit tuple isomorphisms for boilerplate code like that.

<@pkhuong> Oct 9 For sure. Much easier than writing our all 6 comparator methods by hand (:

I assume you also hash in the class name :p

I’ve always thought that hash methods should return an opaque hash_t type, so you aren’t even tempted to do this kind of crap.

<@pervognsen> Oct 9 I think the right interface for extensible hashing is closer to serialising data to a stream (that happens to hash its input bytes). I saw a C++ standard proposal like that; I don’t know what happened to it.

<@pervognsen> Oct 9 Yeah, I was about to correct myself. You need that for streaming. Then you just provide variadic helpers for the combiners. And it lets you distinguish internal hash state vs final hash value.

I prefer the “fold/append/mix into hash state” API for a systems programming language since it doesn’t involve other intermediate objects.

Hrm, I vaguely remember the C++ proposal you’re referring to. I don’t think it was ratified, but I recall it used the streaming style.

<@matt_dz> Oct 9 A couple, most recent in 2015 (http://wg21.link/P0029 ) and 2016 (PDF: http://wg21.link/P0199 ); latter withdrawn: https://botondballo.wordpress.com/2016/07/06/trip-report-c-standards-meeting-in-oulu-june-2016/ ….

• MIDI
• x86 ISA
• SQL
• OpenGL
• Any programming language

These are examples of well-defined interfaces which have allowed big components reuse and their independent improvement/optimization.

For example, a programming language is a well defined interface, to reusable components (compilers) creating computing automatons (programs)

Relationship between idempotence and declarativeness. I.e. you can apply f as many times as you want because it encodes something about the end result, not an action to be perform. For pure functions it is easy, for procedures the trade-off is in some form of state retention.

Tadashi Takieda says:

count(Def) < count(Theorems) < count(Examples)

count(Def) < count(LogicalResults) < count(Examples)

https://www.youtube.com/watch?v=J7vojBbvudQ&feature=youtu.be&t=139

“Design Of Everyday Things” ; Affordances ; Design of clean programming interfaces: tools coming from Design, tools coming from Mathematics ; Mathematics is mostly about user interface ; Theories and concepts as user-interfaces

Industrial design has traditionally seen itself as a way to attract and seduce a customer (see “Never Leave Well Enough Alone” by Raymond Loewy) Some amount of surprising arrangements, within the constraints of serving logically the function of an object was therefore necessary. It seems mostly useless and slightly harmful to care about this aspect for internal modules. It could be a factor of success for an opensource or commercial library.

In the book see his example of the egg as an ideal form perfectly adapted to its function. Balance between strength and aerodynamics. The principle of economy (of materials for instance) leads to elegance.

A first principle of construction: on no account allow the engineering to dictate the building’s form . . . . never modify the social spaces to conform to the engineering structure of the building. –Christopher Alexander

Traditional software engineering wisdom says

• Retained/Immediate (Q. about data-retention / automation)
• Push/Pull (Q. about latency / single vs multiple control flows)

https://www.youtube.com/watch?v=oyLBGkS5ICk

There are two types of changes to an interface: grow or break. At every level of what an API provides and requires:

Grow: provide more, require less Break: provide less, require more

Levels:

• artifacts
• names
• functions

https://lwn.net/Articles/336262/

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1967.htm

Darryl Havens (responsible for the I/O system on NT)

“If you spend that amount of time designing something and you have a spec that gives you every single API [application programming interface], what its inputs are, what its outputs are, you pretty much know how the thing is going to work,” Havens said. “So I actually typed in the code for the entire I/O system in three weeks. That’s how well-designed it was. By the time I sat down to write the code, I already pretty much knew how it was going to work.” Read more at https://news.microsoft.com/features/the-engineers-engineer-computer-industry-luminaries-salute-dave-cutlers-five-decade-long-quest-for-quality/#ydU2bC61j8j51kq6.99

https://news.microsoft.com/features/the-engineers-engineer-computer-industry-luminaries-salute-dave-cutlers-five-decade-long-quest-for-quality/#hskvWsfxAHmVR0Qq.97

https://www.amazon.com/API-Design-C-Martin-Reddy/dp/0123850037

Granularity. git has fine and coarse commands.

I use git gui quite regularly to add/commit/amend messages.

Here is a list of commands I perform regularly:

What is the problem domain that git solves for me?

My perspective is that I use git to:

• keep up to date with most recent version of the software I’m working on,
  • ((git fetch ; git rebase or git reset | git pull) ; [git push –force-with-lease])
• make local changes safely w/ annotations (why) and rollback,
  • (git add ; git commit -m | git gui)
• publish changes to automated test systems
  • (git push -u origin <branch-name>)
• prepare changes for review by colleagues,
  • split changes in atomic digestable micro changes
    • (git rebase -i origin/<base-branch> ; emacs + git gui)
  • annotate changes with messages
    • (git commit | git gui)
• publish changes for review by colleagues,
  • (git push –force-with-lease)
• publish changes officially to the codebase (sync to main line, then publish)
  • grab lock for main line
  • keep up to date
  • command line (git merge –no-ff ; git push) | github merge (most often, so I can copy information)
  • <other routine tasks>
  • release main line lock

That problem domain is almost entirely defined by the tool of version control itself, of which many tools exist.

API Patterns: Immediate Mode UI

In the dimensions of API design. Where does the Immediate/Retained axis lives?

We are talking about two API styles. Retained and Immediate APIs. Although their canonical examples are the construction of user interfaces, these styles are adopted in various other domains.

Retained API:

• the user pre-declares entities and their relationships,
• dynamic properties of the system, behaviors are injected as entities, (callbacks, visitors, listeners, controllers) and associated with the entities,
• control flow is spread across the network,
• network of entities must be mutable, to allow to dynamically add a number of entities that are unknown at compile time,
• user owns reference to entities,
• entities must be explicitely synchronized with the application’s data model,

If data is stored on two sides of an API boundary then you are forced to do synchronization whenever mismatches exist. It is worth it when the automation that the API implementation can perform as high-value compared to the complexity and cost of synchronization.

Example of natural retention: databases, filesystems.

Immediate API:

• uses code/function calls to declare and implement effects,
• composition of entities is created using function composition,
• data model is translated into effects and the application reacts to changes directly where these effects are declared
• presumes that the application can know when data has changed, and that it know what to redeclare,
• implementation usually must be able to diff, cache and clip for performance reasons,
• top down control flow that’s easy to reason about (no callbacks),

Note that in case you have 60hz animation, detecting that something has changed on the screen is a non-issue: there’s almost always something changing on the screen. So the question is rather what has changed and how to update it quickly.

Typical problems with immediate mode ui is layouting. I.e. global constraints that connect multiple entities.

Context for retained APIs. Different computer architectures? Did Xerox PARC use retained mode APIs? This can actually be checked at [http://xeroxalto.computerhistory.org/xerox_alto_file_system_archive.html] After reading a bit this is mostly procedural code with state w/ top down control flow. The Laurel email client written in Mesa has something called BuildScreenStructures that reminds me of traditional retained APIs. See http://xeroxalto.computerhistory.org/Indigo/DMS/Laurel/6/.InitInteractor.mesa!1.html

The style that immediate API follows can be summed up with the following slogan: “Leave it to the client” (Butler W. Lampson, Ref.001)

We need to demonstrate using an example.

Ideas:

• C++ native app ALF/Qt/Dear IMGui?
• HTML App

• Ref.001 Hints For ComputerSystem Design by Butler W.Lampson, where he comes up with a series of slogans.

Leave it to the client. As long as it is cheap to pass control back and forth, an interface can combine simplicity, flexibility and high performance by solving only one problem and leaving the rest to the client. For example, many parsers confine themselves to doing context free recognition and call client-supplied “semantic routines” to record the results of the parse. This has obvious advantages over always building a parse tree that the client must traverse to find out what happened.

• Ref.002 https://mollyrocket.com/861

• https://github.com/ocornut/imgui dear imgui,
• https://github.com/vurtun/nuklear
• https://mollyrocket.com/casey/stream_0020.html
• https://mollyrocket.com/casey/stream_0027.html
• http://lambda-the-ultimate.org/node/4561
• http://gamedev.stackexchange.com/questions/24103/immediate-gui-yae-or-nay
• http://c2.com/cgi/wiki?ImmediateModeGui
• https://news.ycombinator.com/item?id=4201748
• http://www.johno.se/book/imgui.html

1. use a module by reading its documentation only,
2. if a doubt arises and you happen to read the implementation, update the documentation

retained vs immediate (gl/dx/imgui/qt) stateful vs stateless (see EJB) heap vs stack

Immediate UI rely on ID to keep track of an element across frames of operation.

“` \Exists c, cT \Suchas cT=Type(c) ∈ Controls: \Forall d ; id(c) = id(d) \implies c = d “`

Most controls have labels. The label can often conveniently be used as an id. Two counter examples:

“` \Exists d, c \Suchas Type(c) ∈ Controls ^ Type(d) ∈ Controls: \Exists f ∈ Frames: label(d, f) = label(c, f) “`

“` \Exists c \Suchas Type(c) ∈ Controls: \Exists f_0, f_1 \Suchas label(c, f0) != label(c, f1) “`

As you cross boundary layers there might be a tendency to notice that:

• from events people create a static representation (snapshot value)
• from multiple values people generate by difference the events that could be observed (because that corresponds more neatly to the work to be done)

“` A_0 (db1) A_1 (db1) -> add/remove events -> B_1 -(diff w/ B_0)-> add/remove operations -> C_1 “`

http://chipmunk-physics.net/forum/viewtopic.php?t=1434 http://twvideo01.ubm-us.net/o1/vault/gdc09/slides/gdc09_insomniac_physics.pdf https://github.com/erwincoumans/PhysicsEffects

In math, functions are often described as: \(f : Domain ⇒ CoDomain\)

In programming, where we have many names, a short nice name like f isn’t available a longer name is needed.

In C-like languages this translates to something like:

Planet x;
double y = PlanetVolumetricMeanRadiusInE6KM(x);

This emerges from a need to find a trade-off between:

1. ease of enumeration of all transformations applied to a Domain (e.g. via auto-complete)
2. ease of identification of the result entities of a transformation (especially for large names)

The former leads us to prefix the function names with their domain to allow auto-completing symbols using the domain type of the function as a starter stem.

In a typical OOP-style we obtain instead:

Planet x;
double y = x.VolumetricMeanRadiusInE6KM();

Here the the result entities are clearly highlighted while the domain is hidden, and auto-complete requires some type-inference or context awareness.

Functions are also analog to maps. What is the map syntax for this?

It is either:

map<Planet, double> VolumetricMeanRadiusInE6KMForPlanet;
auto y = VolumetricMeanRadiusInE6KMForPlanet[planet];

or

map<Planet, Point> PlanetVolumetricMeanRadiusInE6KM;
auto y = PlanetVolumetricMeanRadiusInE6KM[planet];

Trade-offs at play:

• Friction between map representation vs function
• Namespace organization w/ auto-complete vs result entities clearly highlighted

However if there’s one thing where maps aren’t analoguous to C/C++ procedures, it is in that a map only has a single “argument.”

The point of low friction is clearly with the pair:

1. procedure: src_C{y = PlanetVolumetricMeanRadiusInE6KM(x)} and,
2. map: src_C{y = PlanetVolumetricMeanRadiusInE6KM[y]}