Fern Development Roadmap
You are probably here because you want to contribute. If so, make sure to check the Contributor's Notes too. Contributions are always welcome!
Fern will form the groundwork for the rest of the project. In general, to learn about the birds-eye view of the architecture, see The Power of Structure/Fern. Also: Fern Overview + KR and Constraints. Not everything is in those writings, some things are yet to be refined and understood, most likely in the course of the development.
The roadmap is something like this (in no particular order):
Table of Contents
I am going to work on all of KR problems first. Starting with syntax, and then moving on to everything else. When that's done, then would be the time for either 2D API or constraints. I designate the stuff I am working at the moment with the WIP
tag.
Right away, I am going to say: none of these are trivial and most still require some design choices. Although, possibly, the simplest stuff, design-wise, is everything in the 2D graphics department.
Also, these are not in order. Well, except kr and constraint syntax will have to be adjusted before just about anything else is done. That is, indeed, the first thing I will do.
After that, a few sufficiently independent branches of development could be identified:
- The constraint features are fairly independent and probably won't matter much for anything within Fern itself: async execution, change predicates, parameterized pointer variables, advice.
- Meta info via slots, which will include advice. This may influence quite a few things. Yet to be explored.
- Basic cells (where the root cells are just rendered as windows) -> 2D API for geometry | 2D API for text | Event Objects
- Configurations
One thing I haven't listed here is tests. Tests will have to be written for everything. So far, there are just some tests for linear-algebra and geometry.
Before I get into the details, there is one more point I want to address, because you are probably here because you want to help but aren't sure where to start exactly or might just want to try your hand at something small.
Well, the few fairly isolated things I can think of are: change constraints and async execution. These do require the understanding of how constraints work, though. And if not that, then anything in 2D graphics can be a good place to start, too. And then there are some tickets:
https://todo.sr.ht/~project-mage/tickets
In case you do decide to work on something, do tell me and do ask questions! For one, this may avoid duplication of effort. For two, I might provide some help and further details.
1 Knowledge-Representation Improvements WIP
1.1 Better syntax, cleaner API & refactoring DONE
This will be done to look more like what was in the The Power of Structure article, but cleaner. In fact, all of the API can be minified and reduced in various ways.
1.2 Meta info in slots
An example of meta info: the list of local-only slots. There will be some other stuff too, like advice, probably. But garnet provides some other declarations via it's :declare
syntax, and each thing there will have to be looked into carefully, case-by-case.
Right now, that kind of info is held as part of the kr object and has a special interface. It would be better to just hold these within the slots of the object itself. This would simply be more beautiful, so to speak.
How this is done might depend on the meta info itself. The usual :declare syntax (to be ditched) is read before the slots themselves are read. So, losing the order might mean some things might have to be done dynamically now (some already were).
It's yet to be seen how this is done exactly. Generalized selectors might come in handy here. Some meta slots might be needed early on (like local-slots) and so this feature might have to be implemented soon, but at least in part.
1.3 Advice
Advice like in Emacs. That usually implies: adding a pre-hook, or a post-hook, or around-wrapping. It should be easy to inspect the existing advice and remove/redefine it. Emacs uses named advice to that end.
Several kinds of advice were identified (the following + the constraints). The interface should be as uniform as possible.
1.3.1 Slot behaviors
Behaviors are stored in slots of objects. Then you just send a message to the object:
(↑ object :slot-with-behavior arguments...)
So, there might be some other slot that keeps info about the advice for the function receiving a message.
1.3.2 Schema creation/destruction
There should be some way to advice the create-schema/destroy-schema functions per any given instance. Setting up some meta slots with hooks could be enough. For instance, :post-creation-hook
could refer to the :init
function by default.
But pre-creation-hook
needs some more thought.
1.3.3 Slot addition/removal
It should be possible to add these behaviours as slots, like you would add a constraint. And, in general, changing some behavior of an object should come from modifying the contents of one of its slots (excepting external functions).
One way to implement these would be to maintain two slots: one containing the list of all slots, and another one indicating the one added last. Then the user could simply write a constraint for either of those. So, stuff like demons in original KR seems to be unnecessary (See this ticket).
Also, the constraints shouldn't have to require outputs, they should be able to output to nil, indicating they are there for side effects.
1.4 Multi-Methods (value-based)
Clojure does the reasonable thing with its multimethods: it recognizes that it may be really useful to dispatch based on some function of the value, rather than just on its type. I am still thinking through the mechanics of this, but it's clear to me that CLOS/MOP is not to be used for the simple reason of its complexity beyond reason. But I don't find Clojure's approach fully satisfactory either.
As of this moment, I think that a decision tree should be built, where the slots are ordered, and each slot contains a user-supplied decision function. And defining a method would simply add to this tree. Such trees could of course be modeled as schemas, and that will yield a free, easy and inspectable interface to these functions.
Of course, such dispatch won't readily yield good performance, but probably no worse than CLOS. But it can be achieved via declarations where necessary (I am wondering if declare
can be reused for custom stuff, but it's not strictly necessary).
Later on, these functions can be used in constraints as well.
1.5 Generalized selectors QUESTIONABLE
This feature also allows custom containers to be used along/in place of the default hash-map. The runtime option of slot addition/removal should always be preserved though, but that might simply be a requirement for any new user-supplied data structure.
Edit: this would be nice to have, but this complicates things needlessly, even though I have a pretty good idea of how to do it. The parametrizable pointer variables will have to be done seperately anyway, and working with containers can be done without this. So, unless I encounter some really good use-cases, I will assume this feature is not needed. It is quite an entertaining idea, however.
2 Constraint Management Improvements
2.1 Better Syntax & Cleanup DONE
This will be done to look more like what was in the The Power of Structure article.
2.2 Parameterized pointer variables
Requires generalized selectors.
A very important usability feature. How hard this one will be is yet to be seen. At a glance, screwing with the constraint engine shouldn't be necessary.
2.3 Advice (for constraints)
These will have to modify the constraint objects themselves. I imagine the constraint object could hold some kind of a kr object within that would contain all the advice info.
2.4 Store constraints as KR objects
In fact, I am thinking, it might be useful to turn the constraint objects into KR objects. Then you could just store functions in KR slots and advice them like you would advice any other function. This would solve the advice problem, and it would allow constraint inheritence, which is arguably a more serious problem. Currently the constraints are simply copied. Instead they could use the mechanism of contextual inheritence (where the slot isn't copied, but instead the object is instead inherited from the parent slot; the semantics of this is yet to be fully specified).
But doing this would also expose a simple, familiar way of modification and inspection.
2.5 Async execution
Might come in handy for lengthy operations.
If a constraint output is defined as async, then the constraint simply spins off a thread for its execution. Once the execution is over, then the structure is locked and modified.
It should be possible to sync all such outputs, so that their modifications are applied all at once (once they are all finished). In this case, all outputs need to be executed asynchronously.
2.6 Change predicates
Whenever a constraint calculates an output, it will write the value to the output slot, which may trigger further recalculations. This may be wasteful when the output is the same as the previous value.
The user should be able to supply an equality predicate that indicates whether the value is the same as the last one or not.
Syntactically, this will be done like the async stuff, and maybe even like advice: just by storing this type of info in some kr-object of the constraint.
3 2D Graphics
In general, the point here is to develop an API to draw stuff to a texture. SDL2 will guide some of these API decisions, e.g. when building event objects.
3.1 Linear Algebra and Geometry additions
For geometry, that would be KR equivalents of geometric objects.
Linear algebra just needs to be tested in the course of the development, just to see that everything holds up right.
See Fern Overview + KR and Constraints for details.
3.2 2D API for geometry + Backend (via SDL)
3.2.1 Geometry
There will be two kinds of geometrical objects: structs (already implemented) and kr-objects (not yet). KR-objects will contain a struct in one of its slots and then provide other slots for easy manipulation. For example, an axis-aligned rectangle struct may be interfaced by it's right/left/top/bottom/width/height slots. But the struct itself only contains two vectors indicating it's two sides. So, theree will be circular constraints to keep everything consistent. Note that the structs themselves are meant to be used immutably.
There should be operations for working on collections of objects. Maybe not now, but at some point.
3.2.2 SDL2 Backend
The backend that implements the API will be SDL2.
There's an SDL2 CFFI bindings library that I have prepared It's a fork of hu.dwim.sdl, but with lispy names and error-handling and some quality-of-life macros. You can check it out here. Note that you need a custom CFFI to run it (just for now, hopefully).
(Note: I haven't checked the working status of that library for nearly a year, so it might need some refurbishing; but, overall, it's ready.)
3.3 Event objects + Backend (via SDL)
Just kr-objects that hold information about some event like a mouse click or a keypress.
These don't have to be comprehensive, e.g. stuff like joystick input can be safely skipped for now. The window events might not be necessary either: they backend should modify the appropriate properties of the top-level cell directly.
3.4 Text API + Backend (via Pango)
The interface for this will be inspired by Pango. Maybe even simpler (aka skewed or bent text is not exactly a requirement). Only proportional fonts are.
Backend-wise, Pango works well with Cairo, and there's a way to use and SDL surface for Cairo. There is a way to draw Pango directly to an SDL2 texture too, via SDL2_Pango, but the work for that has to be tested as it seems to be pretty new. It shouldn't matter too much for a backend at this point, but the Cairo variant might be easier to install, and is probably better-tested.
4 Cells
This will include contexts, rendering + texture caching, some interplay with the configurations.
Contexts are basically slot-level inheritence relationships, where a slot inherits from some slot in some other object. Should be pretty easy, will probably be done just via some kind of a constraint.
Caching means how you handle textures and drawing.
Configurations: any object added to the tree may have to be autoconfigured.
Shouldn't be anything hairy.
Cells will be done last, or along the course of the development as other features are introduced.
5 Configurations
I have described how these should work in The Power of Structure/Fern/Configurations. Note: I figure the usual inheritence should do the trick if you structure the program correctly, which shouldn't be hard to do. The special kind of inheritence I have described previously (and in the article) is probably not necessary and would only complicate things.
Overall, I don't imagine this should be hard. This be done after everything else is in order.