Data-Supplied Web as a Model of Free Web

Web. Web.

What's wrong with Web?

- Web is just fine. It's just that some sites are bad and need fixing!

Hm. Maybe. Maybe so. But then again, how many sites have you seen that are truly good?

Oh, well, look at it this way: what is web actually suitable for? I think, and that's just my opinion, the blog-type stuff like this site you are reading here is probably where the line gets drawn.¹ And even that's kind of pushing it.

Just one step beyond and you are suddenly using copious amounts of JavaScript and doing all that "backend" stuff. And then the "frontend" stuff.

Good for you, good for you…

But me, a user? No, fuck me.

Good Web ends where JS begins.

Why? No, it's not just because some foreign code could be malicious. No, the reason is:

I can't modify anything.

That's the one problem with modern Web for the user, and, especially, the power user.

Think of all those low-calorie, UI-heavy websites. And then the sites with lots of data, but non-existent UI. And anything in between.

Those are hard to use. Moreover, modern web forces HTML and CSS on you. That inevitably leads to bloat, as I have argued in another article.

How do we solve this?

I don't think we can, not to the full extent anyway. Web is so commercialized and shitty that good luck fixing that.

But there are websites that are made for the public good.

By the people, for the people.

And those don't have to be shit.

What could there be to make them good?

The Free web. The data-supplied web.² Note that "web" in "data-supplied web" is not a misnomer. I like to think it refers to the general idea of "web", not the "World Wide Web" or its particular implementation.

The purpose of a data-supplied website is to supply data. And, perhaps, to suggest a program for working with that data.

So the point here is that the user gets whatever program he wants to work with the data exposed by a data-supplied website. Perhaps, it's not even a website-proper at this point, but just a data-supplied source on the internet.

- Yeah, no shit, there are websites out there that provide an API for their data.

Yes, yes. I didn't say this was something new! But there's still a need specify a few functions that would make the experience really proper. Here are the functions that a data-supplied source would provide:

Meta

Get technical meta information about the data source: it's sign-up and authorization format; the suggested programs to work with the source.

This is human-readable text and is supposed to instruct how the client programs should be written to work with the source correctly.

Register

Register on the source. (Optional.)

Authorize

Some connection protocol/routine for authorized access. (Optional.)

Get

Get data from the source. This doesn't necessarily get everything. Just the data you ask for.

Post

Modify some data on the website.

Subscribe

Subscribe to data modifications. Could be a specific subset of data.

So, whatever the data format is, every site describes some kind of an access scheme to the data (and this is described as documentation in meta for the potential user to read and understand).

The argument to get, post or subscribe could be as complex as necessary, but it is constructed by the user according to the specification of the scheme. So, access may be partial, which is especially useful when there's a lot of data involved.

It's worth noting that the server can treat certain places in the data as objects and the post request by the user may relay messages to those objects.

At last, the user can cache whatever data is interesting and then subscribe to the changes of that data.

So, to get this summarized:

• There are no restrictions on either the data format nor the kind of programs that can communicate with the source.
• The UI and data are kept separate.
• The user is in full control of the UI (and not the other way around).
• The user can write programs that modify the data and post it back to the website. The user could also just call foreign functions via the same method.

The most problematic part of the experience, for the user, is then to find and install the program that communicates with the website.

The best part of the experience, for the user, is then to find the right program and modify it to his own needs. Write a new one from scratch even. Or, hell, just work with the data directly in a Lisp machine, or something like Kraken! (See The Power of Structure.)

If the source allows modifications, the user program could have appropriate editing capabilities, and if the user is authorized to make modifications, then the website will accept them via a post call.

Generally, a web-supplying API could be useful for web sources that have a lot of data, but the authors realize that writing a complex browser-based UI is bad for your health (and theirs). There are more sites like that than you might think: forums, chats, search engines, dictionaries, trackers, maps, file-hosting services, all sorts of databases.³ Too bad many of these have to show you the ads. But monetizing an API is not such a rare practice and is feasible as an alternative.

But the websites aren't the end of it either. In The Power of Structure, I have discussed Hero, a distributed knowledge tracker that I want to build as part of the Mage campaign. The server for Hero will specify how to access and modify any part of the knowledge network: it will be data-supplying.

So, you can build distributed applications too.

—

And, well, look at the results now: you can mix and match whatever data you want (and from many sources); analyze, modify and post it in-place; write your own workflows; integrate it into other software; cache anything you like; automatically pick up on any interesting changes.

Now, that's speed. That's choice. That's freedom.

And that's what it was supposed to be like to begin with. Data and people picking it apart the way they desire.

Every time GitHub or other large corpo just literally moves its left toe, I see users threatening to run somewhere else (or self-host).

Well, these centralized systems control both data and UI. And the users fear because they can lose data. Just as badly, they can't do anything about the UI.

So, there's a rather interesting issue with the web at large: you may be running a free JavaScript interpreter in a free browser, on a free OS, and even on some free hardware, but surfing the web is yet to become a free experience. Meaning of free: freedom. The freedom to mold your experience with a given entity by accessing the blueprints that let you work with it.

A modern website, if it is to become free, must decouple the notion of data and the program that operates on it; leave the choice of the program to the user, and only expose the data and a good access scheme for that data.

CSS and HTML are not the bare necessities. Data is. Good UI is. (Speaking of which, a truly good interface can only be done as part of an extensible environment. And our browsers aren't that, and won't be for the foreseeable future.)

Like I mentioned in On Flexibility and Software Temples, there's this idea that operating systems should be programmable virtual machines that run a certain language.

An OS as an interactive image that runs independent services that communicate via message-passing.

And, I think, from the user's point of view, this idea provides a very reasonable way to achieve some scalable grounds for the data-supplied web.

With the right kind of privilege and separation concepts, the user could auto-install any program suggested by the source, without any risk, and that would eliminate quite a barrier for adoption of the data-supplying web model.

(Say, in Common Lisp, that could possibly be achieved with a notion of first-class environments. And, perhaps, with some way to set privileges for these environments.)

Otherwise, without such a system in place, the user has to take a risk and assume caution when installing programs, and since there's so many websites, obviously, data-supplied web can't really become safe or easily-browseable at large, certainly not until the right kind of bare-metal operating systems or images come along.

But from the perspective of a power user, for now, even a few websites or applications could make a lot of difference.

And then, of course, browsers are themselves being likened to operating systems. Walled gardens, with no efficiency to speak of, with very little freedom, with too much leverage from the browser vendors. A perfect exploitation machine for keeping you within itself, all while it will do anything to harvest information about your activities, so it can show you some ads as soon as it can. An operating system alright. Yeah, just relax and no harm will come to you.

• ← Overcoming the Print-Statement
• Isn't It Obvious That C Programmers Wrote Git? →

• Table of Contents#
• Top#
• All Essays