Overcoming the Print-Statement

(Published: 12 January 2023)

A process cannot be understood by stopping it. Understanding must move with the flow of the process, must join it and flow with it.


Pain and Simple?

For the purposes of debugging… say, what would it take you, in any program you are developing right now, to observe some number-valued variable by graphically plotting it over time as the program is running (in real time)? You can recompile the program. Pick any variable you like, choose any tools that make your job easiest.

Just think for a minute what you would have to go through to accomplish that: plotting a variable as the program is running.

Step by step, what did you just do?

OK, now. Was that simple?

But it's the simplest of things to ask for.

Just think about it. One variable. One plot. And how much trouble you would have to go through to get it.

If you can't do that kind of thing on a whim, there's a huge problem hiding somewhere.

And if you did it rather painlessly for one reason or another: Is your original code any better for it? Or did you just get yourself a maintenance headache? If you decided to use an external program via a file output, how's the performance faring? Or did you just import a graphics library into your project?

A Thing in Itself

All the sophisticated debuggers in the world haven't overtaken the one puny print statement.

And there's a good reason for that. When you are debugging a program, you want to observe the behavior of your program over time, in multiple places, and a lot of it. Sometimes, from multiple sources, like in the case of multithreaded programming.

Debuggers only let you be in the present, in one thread, with an option to just go forward, one step at a time. No wonder everybody is still using print statements.

Print statements are universal. A universally-accepted hack to get something on screen as fast as possible. Maybe lots of it.

It's pretty clear that debuggers and print statements simply have different scope and capabilities, and it's pretty pointless to try to prove that one is "better" than another.

However, there's a discrepancy: we have pretty sophisticated debuggers out there, but the print statement? It's the same one everywhere.

Well, some folks also write "proper" logs into a file, and that's fine, although, I don't feel logs have much of a qualitative advantage over the print statement. They are mostly a postmortem kind of deal anyway.

There's still a third way to debug: and that's to write a specialized tool. If you are building a game, you will likely just draw some debugging information within your game.

But the print statement still hasn't gone anywhere. You have probably written a few while you were making the specialized tool, and retained a couple just in case. Probably commented out a bunch of them.

Print statements are ad hoc. And, damn, are they indispensable.

Just as they are damn hard to work with and with just as much overhead of all kinds.

There's little that can make you feel so wretched as when sifting through a bunch of console output, cursing at the formatting and trying to figure out where some value has come from, and when. Maybe you are beginning to think how you have gotten yourself into this mess in the first place.

Some print statements go beyond the purposes of immediate debugging: we also tend to output the results that just seem useful in some way. But also, there are all kinds of warnings and, perhaps, some status information, or a rare error condition.

All kinds of structural information.

All of that data gets converted to a string, of course, and ends up in the standard output. Where else could it go!

Well, sometimes your logging framework will stash some information away nicely, limit the madness a little bit. But it has just soothed the symptoms.

It hasn't healed you. It hasn't healed your program. In all fairness, it might have gotten you just another kind of a headache.

Log some variable over a period of time. Take a look at that output now. Scroll it. You are using some form of a buffer. Or a terminal. It's probably hard to work with.

Log a variable. Another one. Get curious about their relationship. Plot them against each other… Why is it all so damn hard and messy to do?

Now What?

We need a way out of this. A good way, if we are to respect ourselves. Not some incremental roundabout.

And, you know, if you have read some of the other writings on this website, you are probably getting tired of me bitching about how structureless everything is. Because I am about to tell it to you again: print statements aren't your friends. And your logging tools are silly.

Even worse, you have probably riddled your code with all the debugging code you now have to maintain.

We need a new kind of tool: the kind which will help us avoid the downfalls of the print statement, while still retaining its ease of operation.

Really, the problem of debugging can be seen like this:

  • your program is generating some information,
  • you want to know where in your code that information comes from, without specifying this in the code,
  • you want an easy way to work with that information.

And so: the output of your program needs a tool separate from your program.

That tool has to work in real time, it has to be interactive. If your program crashes, your tool has to keep working. And it has to know where in your code the output is coming from.

That tool must build a correspondence between code and the output that the code is generating.

That tool must also be programmable. Powerful enough for you to program it to visualize or process anything you want, without ever touching your original program. In fact that tool must be powerful enough to build custom interfaces, for whatever use cases of output you need to process.

Well, that tool has to be your IDE (development environment).

Well, maybe not your-your IDE, but just some hypothetical IDE that's capable of all those things.

Only an IDE can build a runtime correspondence to your code and let you invisibly slip in those output statements where you want them. That IDE is supposed to understand the structure of your code, so that the output is automatically marked with its origin in the source.

Correspondence to the Code Structure

So, obviously the IDE has to be pretty powerful. In fact, it has to be built on top of a GUI which you can use to work with the output of your program. It also has to know about the structure of your program.

And, by the way, what I am describing here can apply to any language, really, whether compiled or not. For the purposes of this article, we will just stick to Common Lisp, though.

How I imagine it working is pretty simple.

Suppose you have this function:

(defun 3n+1 (n)
  (if (= n 1)
      (3n+1 (if (oddp n)
                (+ (* 3 n) 1)
                (/ n 2)))))

If a number n is odd, return 3n + 1, else n / 2. It's conjectured that doing this repetitively always yields 1.1 Don't try this at home.

Calling (3n+1 7) will generate a path for n:

22 11 34 17 52 26 13 40 20 10 5 16 8 4 2 1

So, the function just tests whether this conjecture is true for a given number (by the fact of it returning).

If we want to trace the actual path of the number, we need to inject the output function like this: (out (if (oddp n) ...).

But with our new hypothetical IDE in hand, we will instead just mark the whole expression for output:2 Sly, in Emacs, does this using stickers. Stickers are a great idea. Interfacing their output is a UI disaster, no less. I am not blaming the author: you probably can't do it much better in Emacs. Emacs is not good at interfaces. Simple as.

(defun 3n+1 (n)
  (if (= n 1)
      (3n+1 (if (oddp n)
                (+ (* 3 n) 1)
                (/ n 2)))))

Upon recompiling the function, the IDE will simply wrap the marked expression in a function that communicates the value to the IDE and then returns the value. So the blue expression is passed through out, which is defined to be something like this:

(defun out (x) (send-to-ide x) x).3 Note that for some languages the IDE can supply the IDE-related function itself, there's no necessity to add a dependency to your project.

Now, where is that output going to end up exactly?

Well, since the IDE is aware of the code structure, it can label the output with meta information about its origins. The expressions have unique identifiers, which can persist over modification and even cut-and-paste operations.

Meta information can also include the function identifier that the expression is a part of. And, surely, we could track the function itself just as easily4 By effectively wrapping its whole body.:

(defun 3n+1 (n)
  (if (= n 1)
      (3n+1 (if (oddp n)
                (+ (* 3 n) 1)
                (/ n 2)))))

So, whenever 3n+1 is called, your IDE will receive the information about this and, in addition, observe the inner expression. Naturally, the IDE may automatically pick up on any hierarchical relationships of the observed expressions.

The Right-Hand Side (RHS)

To view this information, I like to think of something I call the right-hand side (RHS).

Software projects tend to have some limit on the number of characters per line. This greatly helps readability.

I find 80-chars-per-line to be quite pleasant for Lisp code.

The outcome is that the right-hand side of the source code is always empty. I think it's the perfect place to observe your output.

Well, the output may spatially correspond to the observed expressions.

(defun 3n+1 (n) <--- 80 cols ---> | call #16, main thread
  (if (= n 1)                     |
      1                           |
      (3n+1 (if (oddp n)          |  [...] 10 5 16 8 4 2 1
                (+ (* 3 n) 1)     |
                (/ n 2)))))       |

(Note that the stuff to the right of the vertical separator simply corresponds to the code on the left, but it is not a part of the editing workflow (in the hypothecial IDE, you would need to switch to it explicitly to interact with it).)

One thing to note here is that since multiple pieces of code may produce a lot of output, some output may overlap. So, it may be wise to prioritize those parts that correspond to the cursor position.

And, then, why be limited to text? If your IDE allows you to draw custom graphics, you can visualize the information and display plots on the right side as well.

The right-hand side becomes an application of its own.

And if multiple expressions have a lot of output, focusing on one of them will overshadow the rest.

Now, wouldn't that be cool?

One feature that would be handy is time traveling. It's imperative to have the ability to view all your old output whenever you like, perhaps with correspondence to the code that generated it (if the history of its changes is kept).

So, the output can be timestamped.

Also, you could snapshot the important results and values, and cycle between them.

RHS is Interactive

A few new venues open up when you build your IDE with a goal of eliminating the print statement.

The output doesn't just end up being visually present on the screen. You have the programmatic access to all of it. You can tell your IDE to name the output from your observed expressions how you like. Now you can work with named output.

The right-hand side then builds up a knowledge base about the output of your code over time. And, perhaps, something like Knowledge-Representation5 See The Power of Structure. could act as the structure to keep all of that information in an hierarchical fashion.6 Of note here may also be the fact that the observed statements may themselves set and unset states when they are invoked and when they return – this allows for building up hierarchical information that might not otherwise be apparent from the structure of your code.

You can, of course, access that structure for further analysis. Or, well, just switch to the RHS and produce further calculations in-place.

All kinds of interesting output can come from your program, some of which you may want to process further. That's kind of the whole point of this: what you can do with your output once you have it.

So, the Right-Hand Side isn't meant to be just some prettified dashboard (admittedly, a fancy one), but rather a starting point for further explorations with the full power of an environment that is your IDE.

Division of Responsibility

How does this approach compare to the conventional logging/print-statement practices?

Well, here, the IDE takes on the responsibility of all the output processing.

This offloads all that mess from the program being developed.

You have successfully decoupled the two things that don't need to be together.

Often times, the program that is being developed takes the responsibility of processing all the output for debugging purposes. This makes people hesitant to write debugging code, because it inevitably ends up being ugly, and it's too hard to make it worth it anyway when your program doesn't have access to all the interesting tooling such as visualization and statistical libraries. (And the point about hesitancy stands true even if you are using a Smalltalk-like environment where your program is your IDE: there still has to be a mechanism of visual correspondence between the code and the output.)

Some Use-Cases

Let's see at a few real-world examples to understand what we gain.

Multiple Threads

Suppose you are developing a multithreaded program.

Debugging those can be a real pain.

With RHS, you can now mix-and-match the outputs of any given code within separate processes/threads by timestamps. This is accomplished simply by tagging the output with the thread identifier.

This applies to systems with multiple process as well, and to the network programs.

Simultaneous analysis of data from multiple sources can be quite essential to understand some problems.

Game Development

Perhaps, one of the common usage patterns may be that of cyclical output.

One of the vivid examples of this comes from game development. Every game runs at a certain frame rate and it's very often that you need to track multiple values over time. Sometimes you need to inspect thousands of frames to see what has gone wrong and at what exact moment.

Well, now you can easily encode that frame cycle in your output and simply travel between those frames. Hell, you could send object positions and quickly draw up a poor-man's visualization for any interesting element you might find useful.

You might even find it handy to send a binary value like an image, a screenshot.

Moreover, head-on printing or file-logging your output can be hellaciously slow, and that simply doesn't work for games. You need a server running somewhere on your computer for that.

On the other hand, the IDE a) wouldn't have to print it, and b) can process the output in parallel.

Cyclical nature of things turns up all the time in programming: besides the usual for loops, you also rerun your programs and restart services, and all the while you might want to be comparing some values between the runs.

You could write specialized routines to show the differences between successive frames.


Time measurements within your program are just another kind of output.

And since the IDE can recognize hierarchical output, it can naturally build the so-called flamegraphs on the fly.

Some Other Frequent Uses

In the pursuit of eradicating print-statement debugging and logging, we might have lost sight of a few rather mundane things.

The foremost one is: evaluating a piece of code and seeing its return value. Well, that value needs to stick visually right there next to that piece of code! You can come back later, and it will still be there.

Well, some function may get into an infinite loop, or a long computation, and, so, RHS can indicate a running process (and its ongoing output, of course).

Also, when writing macros, you get to macroexpand some form rather frequently. In that case, code is produced. Well, this code may as well end up in an editor of its own, again, on the right-hand side.

What about documentation? Well, why not display it there as well? Say, when your cursor hits the function name, it can come up automatically, or with a binding.

And, of course, actual correspondence to code isn't always necessary or sufficient. In many cases beyond trying to understand what an isolated part of code is doing, you just might want to send some output to a specific place like some data structure or a variable within IDE. You might make it still show up on the RHS and have correspondence to a piece of code, but it's not strictly necessary. You are in full control of RHS, and you are in full control of where the output data goes, and how and where you view or work with it.

You can display the output information wherever you like, not just on the right side of your code. That's a handy perk for usability design, but not much more than that.

The main line of thinking here is all about the fact of separation of concerns, all the while retaining the correspondence to the code that produces the output.

In Summary

So, here is what we achieve with our new (as yet, hypothetical) IDE:

  • We have decoupled the output logic from the application and have offloaded all the responsibility of working with the output to a separate program – the IDE.
  • The act of observing an expression is non-intrusive. You can just color code the parts which interest you and the IDE takes care of the rest (upon recompiling that piece of code).
  • You can forget about tracing the correspondence of output to your code – it's always visually apparent, and it's apparent structurally for programmatic access (with unique IDs or named output).
  • Output is never lost, even if the developed program crashes. You can also save anything to the hard drive and navigate by session later.
  • You should be able to time-travel and see different points of some expression's output history.
  • No need to remove debugging code or riddle your codebase with any debugging code. Your program lacks visualization libraries? The IDE will have them. The program-in-development never has to import anything new (not explicitly anyway).
  • Easy processing of the output. All of it is stored in an easy-to-access structure (e.g. Knowledge Representation).

And, of course:

  • We will have ditched that disaster of a console/terminal/logging-library creation that you are using.

We have also wound up putting all the empty space on the right-hand side of the code to good use. But more importantly, we didn't just make it into a glass window - it is interactive and programmable.

Programmers tend to avoid writing any debugging code until some shit breaks in a bad way and turns into a serious headache.

Writing extra code in an already large code base "just to see" some value (and just for you to remove it later) is less than ideal of an exercise.

There's a big usability gap between program state and the programmer visualizing and working with it. We can easily bridge that gap by building better IDEs.


To achieve maximal utility, the new IDE should be able to track identifiers and expressions across sessions, or at least be able to reconstruct them in a robust manner. In other words, it really helps if the IDE is structural (but it's not necessary).

Also, the output has to be serialized and sent via something like TCP/IP or some other means of interprocess communication.

Probably it's worth emphasizing again that the outlook laid out in this article is not specific to Lisps. Surely, being expression-based helps, but you could do very similar things with other languages.

However, the IDE has to be built from the ground up to support writing custom tools on the fly. And it should be a toolkit in its own right, a hackable environment; an interactive runtime is necessary. Building a tool to help you understand your output should be like molding clay. Or like playing with sand and water, with some cement in places. But not like chiseling marble boulders.

(I will try to accomplish the vision of fluid, sand-like value observation in Alchemy. See The Power of Structure.)



Don't try this at home.


Sly, in Emacs, does this using stickers. Stickers are a great idea. Interfacing their output is a UI disaster, no less. I am not blaming the author: you probably can't do it much better in Emacs. Emacs is not good at interfaces. Simple as.


Note that for some languages the IDE can supply the IDE-related function itself, there's no necessity to add a dependency to your project.


By effectively wrapping its whole body.


Of note here may also be the fact that the observed statements may themselves set and unset states when they are invoked and when they return – this allows for building up hierarchical information that might not otherwise be apparent from the structure of your code.

...proudly created, delivered and presented to you by: some-mthfka. Mthfka!