I wanted to share an observation I’ve seen on the way the latest computer systems work. I swear this isn’t an AI hype train post 😅
I’m seeing more and more computer systems these days use usage data or internal metrics to be able to automatically adapt how they run, and I get the feeling that this is a sort of new computing paradigm that has been enabled by the increased modularity of modern computer systems.
First off, I would classify us being in a sort of “second-generation” of computing. The first computers in the 80s and 90s were fairly basic, user programs were often written in C/Assembly, and often ran directly in ring 0 of CPUs. Leading up to the year 2000, there were a lot of advancements and technology adoption in creating more modular computers. Stuff like microkernels, MMUs, higher-level languages with memory management runtimes, and the rise of modular programming in languages like Java and Python. This allowed computer systems to become much more advanced, as the new abstractions available allowed computer programs to reuse code and be a lot more ambitious. We are well into this era now, with VMs and Docker containers taking over computer infrastructure, and modern programming depending on software packages, like you see with NPM and Cargo.
So we’re still in this “modularity” era of computing, where you can reuse code and even have microservices sharing data with each other, but often the amount of data individual computer systems have access to is relatively limited.
More recently, I think we’re seeing the beginning of “data-driven” computing, which uses observability and control loops to run better and self-manage.
I see a lot of recent examples of this:
- Service orchestrators like Linux-systemd and Kubernetes that monitor the status and performance of services they own, and use that data for self-healing and to optimize how and where those services run.
- Centralized data collection systems for microservices, which often include automated alerts and control loops. You see a lot of new systems like this, including Splunk, OpenTelemetry, and Pyroscope, as well as internal data collection systems in all of the big cloud vendors. These systems are all trying to centralize as much data as possible about how services run, not just including logs and metrics, but also more low-level data like execution-traces and CPU/RAM profiling data.
- Hardware metrics in a lot of modern hardware. Before 2010, you were lucky if your hardware reported clock speeds and temperature for hardware components. Nowadays, it seems like hardware components are overflowing with data. Every CPU core now not only reports temperature, but also power usage. You see similar things on GPUs too, and tools like nvitop are critical for modern GPGPU operations. Nowadays, even individual RAM DIMMs report temperature data. The most impressive thing is that now CPUs even use their own internal metrics, like temperature, silicon quality, and power usage, in order to run more efficiently, like you see with AMD’s CPPC system.
- Of source, I said this wasn’t an AI hype post, but I think the use of neural networks to enhance user interfaces is definitely a part of this. The way that social media uses neural networks to change what is shown to the user, the upcoming “AI search” in Windows, and the way that all this usage data is fed back into neural networks makes me think that even user-facing computer systems will start to adapt to changing conditions using data science.
I have been kind of thinking about this “trend” for a while, but this announcement that ACPI is now adding hardware health telemetry inspired me to finally write up a bit of a description of this idea.
What do people think? Have other people seen the trend for self-adapting systems like this? Is this an oversimplification on computer engineering?
Perhaps I’m being dense and coffee hasn’t kicked in yet, but I fail to see where is this new computing paradigm that’s mentioned in the title.
From their inception, computers have been used to plug in sensors, collect their values, and use them to compute stuff and things. For decades each and every single consumer-grade laptop has adaptive active cooling, which means spinning fans and throttling down CPUs when sensors report values over a threshold. One of the most basic aspects of programming is checking if a memory allocation was successful, and otherwise handle an out-of-memory scenario. Updating app states when network connections go up or down is also a very basic feature. Concepts like retries, jitter, exponential back off have become basic features provided by dedicated modules. From the start Docker provided support for health checks, which is basically am endpoint designed to be probed periodically. There are also canary tests to check if services are reachable and usable.
These exist for decades. This stuff has been done in production software since the 90s.
Where’s the novelty?
Those are good points.
Control loops have of course always existed in industrial computing, but I think it’s exceptional how common they are now in modern servers and PCs.
Thats a good point about memory allocation. I guess a lot of syscalls could be considered to be part of this data-centric self-adaptation mode of operation that I’m trying to describe.
I think retries and exponential backoff are more of a single-threaded error-handling operation, I think that’s different from the operations I’m describing, which instead involves multiple services communicating together to adapt to changing conditions.
As far as I can tell, Docker didn’t add healthchecks until 1.12 in 2016. I do think Docker healthchecks are a good example of the service orchestration design that has become very popular recently, though.
To be fair, I didn’t start seriously programming until around 2017, so maybe I’m missing some of the history that shows that this sort of data-centric adaptation was popular prior to 2010.
I think what this person is saying is that systems and services have been monitored for metrics and logs for a long time, I know I have been doing it for more than 20 years across many OS, hardware platform, and software stack. The tools and depth of the integrations have definitely changed and gotten way better and more sophisticated but I definitely made systems that monitored and healed themselves to varying levels of efficiency since at least using Nagios in 2003 (I’m getting Perl PTSD flashbacks now hah).
One thing that has definitely gotten better in the last 5 or so years though is code level instrumentation and tracing as well as the higher level correlation tools. I have also seen more developers and vendors way more willing to implement monitoring features in their code from the beginning leading to more data and less duct tape and guessing which has been FANTASTIC.
Anyway, great post though, the monitoring arena is definitely way more awesome than ever before these days that is for sure.