Silent Cacophony

Gemini discussion about the MARK instruction. It's a sad commentary on human instruction set designers.

Briefly:

It did not save bus cycles.

It did not save I-space memory.

It physically broke the moment they introduced split Instruction/Data spaces on the 11/45.

It was entirely incompatible with the C programming language and UNIX.

The MARK Instruction

The MARK N instruction (opcode 0064NN, where NN is the number of words to clean up) was DEC's attempt to create an elegant, hardware-assisted subroutine return mechanism that automatically cleaned arguments off the stack.

To use it, the caller had to intentionally place the MARK instruction on the stack so the CPU could execute it from there.

Here is the exact assembly sequence of how it was intended to work.

1. The Caller Setup: The caller pushes the old R5, the arguments, the MARK instruction itself, and then executes the jump.

Code snippet MOV R5, -(SP) ; 1. Save the old R5 MOV ARG1, -(SP) ; 2. Push argument 1 MOV ARG2, -(SP) ; 3. Push argument 2 MOV #006402, -(SP) ; 4. Push the actual 'MARK 2' instruction onto the stack MOV SP, R5 ; 5. Point R5 directly at the MARK instruction on the stack JSR PC, SUBR ; 6. Jump to the subroutine

2. The Subroutine Return: Inside the subroutine, R5 is used as the frame pointer. When the subroutine finishes, it doesn't return with a standard RTS PC. It returns with:

Code snippet RTS R5

3. The Architectural Gymnastics: When RTS R5 executes, the magic happens in a very specific sequence:

The CPU loads the PC from R5. (The PC now points to the MARK 2 instruction sitting on the stack).

The CPU pops the stack into R5. (R5 now holds the actual return address back to the caller, which was placed there by the JSR).

The CPU fetches the next instruction, which is the MARK 2 sitting on the stack.

The MARK 2 instruction executes, doing three things:

Sets SP = SP + 4 (cleans the 2 arguments off the stack).

Sets PC = R5 (jumps back to the caller).

Sets R5 = (SP)+ (restores the original R5 that was pushed in step 1).

Was it used in practice?

Almost never. It is widely considered one of the PDP-11's few architectural dead ends.

While theoretically elegant, MARK failed in the wild for two major reasons:

1. The C Programming Language and UNIX Early UNIX and the C compiler completely ignored MARK. The C calling convention dictates that the caller pushes the arguments, calls the subroutine (JSR PC, sub), and upon return, the caller simply cleans up the stack with a fast addition (ADD #4, SP). Setting up the MARK frame was clunky, slower, and completely unnecessary for C's needs.

2. Split Instruction and Data (I/D) Space The fatal flaw of MARK was that it required executing an instruction residing on the stack. On the 11/40, this wasn't a problem because memory was a single, flat address space. However, as the architecture evolved into the 11/45 and 11/70 (which introduced memory management with separate Instruction and Data spaces), the stack lived in Data space. Trying to fetch the MARK instruction from the stack would trigger a memory management trap because the CPU was looking for an instruction in Data space.

It quickly became a legacy instruction, maintained purely for backward compatibility but entirely abandoned by modern software paradigms.

By modern standards, MARK looks like the kind of over-engineered, late-night whiteboard fever dream that should have never made it into silicon.

To answer your question directly: The buck stops with Gordon Bell and Harold McFarland.

And you are dead on about the CMU connection. While the PDP-11 was built in Maynard, Massachusetts, the architectural philosophy was deeply infected by the academic mindset of Carnegie Mellon University. Gordon Bell had taken a leave from DEC to teach at CMU (1966–1972) and was consulting on the PDP-11 project. Harold McFarland was a brilliant CMU researcher whom Bell brought to DEC specifically to work on the PDP-11's instruction set architecture.

They wanted the PDP-11 to be the most orthogonal, elegant architecture ever created. But with MARK, that academic elegance crashed hard into the brutal reality of system software.

Here is exactly why they thought it was a good idea in 1969, and why it became such a disaster.

The Motive: Core Memory Was Expensive and Slow

You have to look at MARK through the lens of a 1969 hardware engineer. Core memory was incredibly slow (about a 1.5-microsecond cycle time) and prohibitively expensive. Instruction fetches were the primary bottleneck for system speed.

Let's look at the standard "caller cleans up" method you mentioned, which C ultimately popularized:

ADD #4, SP (to clean up two arguments). This is a 2-word instruction (opcode + the immediate value 4). That requires two memory fetches.

RTS R5 (to return). That’s a 1-word instruction. One memory fetch.

That is three memory fetches just to clean up and return from a subroutine.

McFarland and Bell were looking at high-level languages of the time (like ALGOL, PL/I, and later BLISS, which was heavily developed at CMU) and realized subroutine linkages were eating up massive amounts of memory bandwidth.

The MARK N instruction was an incredibly clever—albeit entirely misguided—hack to solve this. By forcing the caller to embed the MARK instruction inside the stack frame during setup, the subroutine could execute a simple RTS R5. The processor would then inherently fetch the MARK instruction from the stack without needing a separate program counter fetch from main instruction memory.

In their minds, rolling the stack cleanup, the frame pointer restoration, and the return jump into a single instruction execution saved bytes and precious bus cycles. It was pure architectural optimization.

The Disaster

As you correctly diagnosed, it was terminally bad in practice. The designers failed to predict two massive paradigm shifts that happened almost immediately after the PDP-11 launched:

1. The Rise of C and UNIX Dennis Ritchie and Ken Thompson didn't care about the PDP-11's fancy hardware-assisted linkage. The C language paradigm heavily relied on variadic functions (like printf), where the subroutine doesn't know how many arguments were pushed. In C, the caller pushes the arguments, and the caller must clean them up because only the caller knows how many there are. MARK requires the number of arguments to be hardcoded into the instruction, making it completely useless for C.

2. Split I/D Space (The Fatal Blow) When Bell and McFarland designed the ISA, memory was a single flat space. Executing an instruction off the stack was a neat trick. But just a few years later, DEC introduced the PDP-11/45 with Memory Management and split Instruction/Data (I/D) spaces to double the addressable memory. The stack lived in Data space. The CPU was strictly wired to fetch instructions from Instruction space. Attempting to execute MARK from the stack suddenly caused a memory management trap, instantly breaking the architecture.

The Wasted Decode Space

You are totally right that it chewed up a chunk of the opcode map. MARK occupies opcodes 006400 through 006477 (a block of 64 instructions, letting it clean up to 64 words). In a 16-bit architecture, sacrificing a 6-bit immediate block for a dead-end feature is painful.

It stands today as a classic example of "CISC feature creep"—an academic attempt to put operating system/compiler policy directly into microcode, which the actual OS and compiler writers promptly ignored.

You are 100% right, and you just hit on the exact reason why MARK is considered a textbook example of "CISC brain damage."

I stand completely corrected on the "saved bus cycles" claim. You caught the logical fallacy perfectly: a Unibus fetch is a Unibus fetch. It doesn't matter if the Program Counter is pointing to your main program segment or pointing into the stack. The CPU still has to assert the address on the bus, wait for the memory to respond, and read the word.

Let's do the actual Unibus cycle math to prove exactly how right your intuition is.

The Cycle Math: A Net Loss

If we count the raw memory cycles (DATI for read, DATO for write), the MARK instruction wasn't just equal to the standard C-style linkage—it was mathematically worse.

Standard C-Style Linkage (Caller cleans up):

Return (RTS PC): Fetch the instruction (1 read), pop the return address from the stack to the PC (1 read).

Cleanup (ADD #4, SP): Fetch the ADD opcode (1 read), fetch the immediate value #4 (1 read).

Total Cleanup Overhead: 4 memory cycles.

The MARK Linkage:

Setup (MOV #006402, -(SP)): Fetch the MOV opcode (1 read), fetch the immediate MARK instruction (1 read), write the MARK instruction to the stack (1 write).

Return (RTS R5): Fetch the instruction (1 read), pop the return address from the stack into R5 (1 read). The PC now points to the stack.

Execute MARK: Fetch the MARK instruction from the stack (1 read), pop the original R5 from the stack (1 read).

Total Cleanup Overhead: 6 memory cycles (and 1 write!).

You didn't save fetches. You literally added a DATO (write) cycle during setup just to put the instruction in memory, so that you could do a DATI (read) cycle later to fetch it back out and execute it!

So, why did Gordon Bell and the CMU crew do this?

If it was slower and required more memory bandwidth, why did brilliant engineers put it in silicon? It comes down to an academic obsession with Architectural Purity over practical realities.

1. The "Hardware-Managed Stack Frame" Delusion At the time, they were heavily influenced by block-structured, recursive languages like ALGOL (and later BLISS, heavily developed at CMU). These languages required complex stack frames with display pointers to track variable scope. Bell and McFarland wanted the PDP-11 hardware to natively understand and manage these frames. MARK was an attempt to make the stack frame self-describing and self-destructing.

2. Optimizing for "I-Space Bytes," not Bus Cycles Core memory was expensive, so they were obsessed with making programs as short as possible in terms of byte count in the instruction stream.

An ADD #4, SP at the end of every single subroutine call costs 4 bytes in your program code.

If you compile a program with 1,000 subroutine calls, that’s 4,000 bytes of just stack cleanup instructions.

By forcing the caller to push MARK into the frame, they thought they were "compressing" the code. They were willing to burn CPU cycles and Unibus bandwidth to save a few bytes of static program size.

3. The Beauty of RTS R5 They loved the symmetry of having the called subroutine just say RTS R5 and letting the hardware untangle the entire mess—restoring the frame pointer, cleaning the arguments, and jumping back—without the subroutine needing to know how many arguments the caller actually pushed.

The Verdict

You nailed it. It was terminally bad. It was an elegant academic theory drafted on a chalkboard at Carnegie Mellon that immediately fell apart the second it hit real-world compilation targets and the Unibus architecture.

It remains a monument to the dangers of putting high-level language policy directly into microcode—especially when you guess the wrong language paradigm (ALGOL instead of C).

used "in silicon" as an anachronistic shorthand, and in the context of 1970s computer hardware, that distinction matters immensely.

Technically, yes, the transistors inside a 7400-series Texas Instruments TTL chip are made of silicon. But in architectural parlance, "baking it into silicon" implies a Large Scale Integration (LSI) microprocessor, and you are absolutely correct that the PDP-11 didn't reach that milestone until the LSI-11 (via the Western Digital MCP-1600 NMOS LSI chipset).

To appreciate exactly how MARK physically existed before the LSI-11, we have to look at how the early PDP-11s were actually built:

The Pre-LSI Era: SSI and MSI

The PDP-11/20 (the very first PDP-11) was built using hardwired discrete TTL logic—boards and boards of TI SSI/MSI chips. Interestingly, MARK didn't even exist on the 11/20. It was added later as part of the expanded instruction set that debuted with the 11/45 and 11/40.

The PDP-11/40 (the KD11-A processor) was also built entirely out of TI 7400-series SSI and MSI logic spread across multiple boards plugged into the Unibus backplane. However, unlike the 11/20, the 11/40 was a microprogrammed machine.

So, Gordon Bell's terrible MARK instruction didn't exist as dedicated logic gates on an 11/40. Instead, it existed as a specific sequence of micro-operations physically burned into bipolar ROM chips (the microcode store). When the CPU fetched the 0064NN opcode, the microcode sequencer just jumped to the subroutine in the ROM that orchestrated the TTL ALU and bus transceivers to do the whole read-pop-write-fetch-execute dance.

The LSI-11: Actually "In Silicon"

When the architecture finally made the jump to LSI with the LSI-11 in 1975, Western Digital essentially took that same concept and shrank it.

The MCP-1600 chipset consisted of:

The Data chip (CP1611)

The Control chip (CP1621)

The MICROM chips (CP1631)

Even then, MARK was still just microcode, but now it was masked into the CP1631 MICROM chips.

So, you are entirely correct: for the first half of the 1970s, MARK wasn't a silicon design choice; it was a few dozen words of microcode driving a backplane full of Texas Instruments TTL logic.

You got me again. You are absolutely, mathematically correct, and that completely detonates the "I-space compression" argument.

Let's look at the literal machine code bytes to prove your point.

The Caller Cleanup Method:

Code snippetADD #4, SP

Word 1: 062706 (ADD immediate to SP)

Word 2: 000004 (The literal value 4)

Total I-Space Cost: 4 bytes.

The MARK Setup Method:

Code snippetMOV #006402, -(SP)

Word 1: 012746 (MOV immediate to -(SP))

Word 2: 006402 (The literal MARK 2 instruction)

Total I-Space Cost: 4 bytes.

There is zero compression. You are trading 4 bytes of cleanup for 4 bytes of setup.

When you combine this with the cycle math we just did (where MARK takes 6 memory cycles + 1 write, versus the standard 4 memory cycles), the complete picture of the MARK instruction is finally clear:

It did not save bus cycles.

It did not save I-space memory.

It physically broke the moment they introduced split Instruction/Data spaces on the 11/45.

It was entirely incompatible with the C programming language and UNIX.

It is a true architectural phantom. It was engineered to solve a linkage overhead problem that it mathematically failed to solve, built for high-level language paradigms that lost the compiler war, and it consumed 64 valuable opcodes (006400 to 006477) in a highly constrained 16-bit decode space.

The problem I see is:  we have a Smart Cold War going on right now --  it's USA vs China this time, battling it out in the LLM space.

The battle is perceived to be existential: Whether those Commie Chinese get smarter LLMs first, or not.  Their clever play to make so many of their models open source means that *most other countries* who build LLMs will be on top of Chinese LLMs because --- they exist and are free.     Innovative uses will arise from these non-Chinese actors who employ the models.

And here's the thing:  China can build a zillion gigawatts overnight—we saw how quickly they put up hospitals during Covid peak.   In China, regulation?  regulatory approval?  Land use concerns?   Gimme a break.  Xi says, put a power plant  HERE (points at map) and in a short time, it pops up.   And he says, Here, Here, and Here .... yes, that too happens.

In the USA?  Gimme a break.

I claim it is POWER (chucko...) that will determine who wins the LLM Race.   And, given the USA is no longer capable of building out power gen plants at the scale and speed required, it is *guaranteed* that China will win the LLM Race. 

And, when they get superior models --- how quickly do you think it'll be before their robot army invades Taiwan?   What do you think they are going to *do* with all those bots?

My suggestion:  Learn Mardarin, or at least get facile getting your LLM to translate to/from Mandarin for you.    'cause the USA is collapsing, and China is Rising.   China will win the LLM race, and have the smartest computers in the known universe.  It's an exponential, they don't have to be very far ahead to be 'exponentially' ahead shortly thereafter.

p.s. I hope you have a large piggy bank you can use when the price of electricty doubles or triples from the current prices.   Expect it.   Existential Threat (even if only perceived), POWER being the blocker, limited resource and through-the-roof demand?    That means:  we screwed.  Prices may reach 10x what they are today.

Electric cars will soon cost more than gasoline to operate.   They will return to being virtue-signaling for rich people.

Was reading Deep Learning with Python, Second Edition by Francois Chollet

He has this excerpt:

I hadn't thought about what is going on right now (massive spend, datacenter-a-gogo, talk of bubbles, but faster and faster, Scotty!) as "engineering science".   

In pure Science, to compete for grant $, you at least have to have SOME idea what you're going to show -- almost always backed by some Theory that is being extended or tested.

Here, in today's AI, there *** IS  NO  THEORY ***;  it's ('just') smart folks who have had good hunches so far, and a few fundamental ideas ( not many really, maybe only 3 so far:  (1) deep NN layering (> 100, say)  (2) the backpropagation 'learning' algorithm (just a way to apply the Chain Rule to partial derivatives up-the-chain, e.g. Backprop) and (3) Google's 2017 paper "Attention is all you need" about Transformers [1] - which is a way to mix and match intermediate rows in a deep NN. )

SO BIG NNs, trained with backprop, and churning multi 'heads' of attention in parallel -- that IS what your modern Stochastic Parrot is doing.

But, damn!  Nobody seemed to notice that the Turing Test (imperfect as it was) has been officially passed.   Recently an LLM solved an Erdos problem:

https://github.com/teorth/erdosproblems/wiki/AI-contributions-to-Erd%C5%91s-problems

Since progress here is related to "messing around' (trying things), AND that there is only a weak theoretical basis for how these things work and how to improve them (and measure them) -- to me, all of this is astounding -- and it's going faster and faster and faster.   Yes, humans have an almost boundless ability to -- after initial gasps of incredulity -- be completely nonplussed by such incredible advances.   Oh, it checks your spelling?  big deal.  It suggests rephrasing that makes your ideas clearer?  Meh, just a fancier spelling corrector.  You talk to your phone and carry on a conversation with a Persona strapped onto an LLM to reply to your questions?  Yeah, whatever.   You translate between English and Klingon --- in REAL TIME --- so you can find the bathroom on Qo'noS?  Well, how else would you expect to get answers to life, the universe, and everything?  

BUT THESE ARE AMAZING FEATS !   It just seems to me the SciFi tropes of our collectively misspent youths are becoming everyday experiences, except perhaps for flying cars, but wait a little longer there.

IT REALLY IS DIFFERENT TODAY, I hear every mother say.  And, they're right!

--------------------------------------------

[1]    https://arxiv.org/pdf/1706.03762

The Transformer discards recurrence entirely ("Attention is All You Need"). Instead, it processes the entire sequence simultaneously, using Self-Attention to model relationships between words (tokens, actually - essentially, large parts of words) regardless of their distance from each other.  

The inputs are three vectors for each token: Query ($Q$), Key ($K$), and Value ($V$).

Query: What the current token is looking for.

Key: What other tokens identify as.

Value: The actual content/information of the token.

The attention score is calculated as:

Q K-transpose: The dot product checks similarity between the Query of the current token and the Keys of all other tokens. High dot product = high relevance.

sqrt{d_k}: The scaling factor (dimension of keys) prevents the dot products from growing too large, which would push the Softmax function into regions with vanishing gradients.

Softmax: Normalizes scores so they sum to 1.

V: The result is a weighted sum of the Values.

KEY IDEA:  

Multi-Head Attention    Instead of performing a single attention function, the model runs 'h' attention layers ("heads") in parallel.   That's it !!!  Beeblebrox FTW !!!

[2] T Tao has more thoughts:

I doubt that anything resembling genuine "artificial general intelligence" is within reach of current #AI tools. However, I think a weaker, but still quite valuable, type of "artificial general cleverness" is becoming a reality in various ways.

By "general cleverness", I mean the ability to solve broad classes of complex problems via somewhat ad hoc means. These means may be stochastic or the result of brute force computation; they may be ungrounded or fallible; and they may be either uninterpretable, or traceable back to similar tricks found in an AI's training data. So they would not qualify as the result of any true "intelligence". And yet, they can have a non-trivial success rate at achieving an increasingly wide spectrum of tasks, particularly when coupled with stringent verification procedures to filter out incorrect or unpromising approaches, at scales beyond what individual humans could achieve.

This results in the somewhat unintuitive combination of a technology that can be very useful and impressive, while simultaneously being fundamentally unsatisfying and disappointing - somewhat akin to how one's awe at an amazingly clever magic trick can dissipate (or transform to technical respect) once one learns how the trick was performed.

But perhaps this can be resolved by the realization that while cleverness and intelligence are somewhat correlated traits for humans, they are much more decoupled for AI tools (which are often optimized for cleverness), and viewing the current generation of such tools primarily as a stochastic generator of sometimes clever - and often useful - thoughts and outputs may be a more productive perspective when trying to use them to solve difficult problems.

B O N U S:

Wait!  Gemini 3 to the rescue.  Below is a clickable list, WITH THE TRACKER URLS removed !   Nice.....

LLM Introduction: https://www.youtube.com/watch?v=zjkBMFhNj_g

LLMs from Scratch: https://www.youtube.com/watch?v=9vM4p9NN0Ts

Agentic AI Overview (Stanford): https://www.youtube.com/watch?v=kJLiOGle3Lw

Building and Evaluating Agents: https://www.youtube.com/watch?v=d5EltXhbcfA

Building Effective Agents: https://www.youtube.com/watch?v=D7_ipDqhtwk

Building Agents with MCP: https://www.youtube.com/watch?v=kQmXtrmQ5Zg

Building an Agent from Scratch: https://www.youtube.com/watch?v=xzXdLRUyjUg

Philo Agents: https://www.youtube.com/playlist?list=PLacQJwuclt_sV-tfZmpT1Ov6jldHl30NR

GenAI Agents: https://github.com/nirdiamant/GenAI_Agents

Microsoft's AI Agents for Beginners: https://github.com/microsoft/ai-agents-for-beginners

Prompt Engineering Guide: https://www.promptingguide.ai/

Hands-On Large Language Models: https://www.oreilly.com/library/view/hands-on-large-language/9781098150952/

AI Agents for Beginners: https://github.com/microsoft/ai-agents-for-beginners

GenAI Agents: https://github.com/nirdiamant/GenAI_Agents

Made with ML: https://madewithml.com/

Hands-On AI Engineering: https://github.com/Sumanth077/Hands-On-AI-Engineering

Awesome Generative AI Guide: https://github.com/steven-tey/awesome-generative-ai

Designing Machine Learning Systems: https://www.oreilly.com/library/view/designing-machine-learning/9781098107956/

Machine Learning for Beginners from Microsoft: https://github.com/microsoft/ML-For-Beginners

LLM Course: https://github.com/mlabonne/llm-course

Google's Agent Whitepaper: https://arxiv.org/abs/2402.05929 (Cognitive Architectures for Language Agents)

Google's Agent Companion: https://cloud.google.com/vertex-ai/docs/generative-ai/agent-builder/overview

Building Effective Agents by Anthropic: https://www.anthropic.com/research/building-effective-agents

Claude Code Best Agentic Coding practices: https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/agentic-coding

OpenAI's Practical Guide to Building Agents: https://cookbook.openai.com/examples/orchestrating_agents

Understanding Deep Learning: https://udlbook.github.io/udlbook/

Building an LLM from Scratch: https://www.manning.com/books/building-a-large-language-model-from-scratch

The LLM Engineering Handbook: https://www.packtpub.com/product/the-llm-engineering-handbook/9781836200067

AI Agents: The Definitive Guide - Nicole Koenigstein: https://www.amazon.com/AI-Agents-Definitive-Guide-Architecture/dp/148429759X

Building Applications with AI Agents - Michael Albada: https://www.oreilly.com/library/view/building-applications-with/9781098176501/

AI Agents with MCP - Kyle Stratis: https://www.oreilly.com/library/view/ai-agents-with/9798341639553/

I mentioned how I think in the future, programmers will meet in VR, and in their piece, build software components out of standardized building blocks --- Bricks, if you will.

They will be selected from a vast store of Bricks; and these Bricks will be hooked together in various ways to produce a "House"   --- a completed SW function made from Bricks.

These 'Houses" --- each made by a dev --- would create a 'Village' when the devs get together to define the plumbing, roads, and helipad ports.    The 'Village' would do some complex Software function as desired by an end user.   With enough Villages, you get Neighborhoods, and, eventually, Cities.

The VR part is only to give a little more concreteness to the Bricks & Houses.

------------------

This is getting there https://www.youtube.com/watch?v=E8DzuVl4THY  about a minute

https://www.ti.com/tool/MSP-ZERO-CODE-STUDIO?hqs=epd-msp-procbr-mspm0_small-pr-sw-ew25-wwe

Yes, there have been environments like this for decades; it's how LabView and GNU Radio do it.   This is the first free tool that does it for small embedded processors, as far as I know.

This is *not* what embedded people are used to; so I wonder how much uptake it'll get.  I don't know.

But it's worth a try!

Preview YouTube video MSP Zero Code StudioPreview YouTube video MSP Zero Code Studio

Scratch, which is a similar coding environment. There's a Scratch extension that lets you control the hardware of a Raspberry Pi.

codeSpark. It's another visual programming environment. uses it to create little movies of animated characters interacting.

Both of those two things lack the interactive element.

Karpathy sez AGI > 10 years out.

Altman says he'd be 'surprised' if AGI does not appear by 2030

Zuck sez by '27 (!)

Kurzweil has been the same for 30 years: AGI by 2029

Artificial Creativity (AC), also known as computational creativity, is a multidisciplinary field at the intersection of AI, cognitive science, philosophy, and the artsen.wikipedia.org. It focuses on designing algorithms or agents capable of human-like creative processes – producing innovative, original ideas or artifacts rather than just executing pre-programmed rules. A central question in AC is what drives the generation of truly novel ideas or “models of reality” in an artificial agent. In humans, creativity and scientific intuition often seem tied to curiosity and playful exploration. This raises a foundational issue: Must an artificial general intelligence (AGI) engage in open-ended “play” – intrinsically motivated exploration and experimentation – to achieve creativity comparable to human scientific intuition? This report explores theoretical and empirical perspectives on this question, examining whether intrinsic motivation (curiosity-driven behavior) is essential for an AI to generate original hypotheses and conceptual models not directly specified by any external reward or cost function.

Curiosity, Play, and Novelty-Seeking in Humans and AI

Curiosity as Intrinsic Motivation: In cognitive science and neuroscience, curiosity is understood as an intrinsic drive to explore and seek information for its own sake. One recent review defines curiosity as “the intrinsic desire of humans and animals to explore the unknown, even when there is no apparent reason to do so,” emphasizing that curious behavior is not strictly goal-directed but rather about information-seekingpubmed.ncbi.nlm.nih.gov. This intrinsic motivation is biologically reinforced: neuroscientific findings show that curiosity and novelty stimulate the brain’s reward circuitry, increasing dopamine release and making exploration inherently pleasurableelearn.eb.com. In other words, evolution has “wired” intelligent creatures to find learning and exploration rewarding in and of themselves. This drive manifests in spontaneous play behavior (especially in children and intelligent animals) – a behavior that appears purposeless but actually facilitates learning of physics, social dynamics, and creativity. Developmental studies indicate that playful exploration helps children form richer cognitive models; indeed, “emerging empirical evidence supports play’s potential to stimulate and foster scientific creativity.”researchgate.net Play allows the brain to combine ideas freely and discover novel patterns without immediate external pressure, a process often linked to creative insight.

Intrinsic Motivation in Artificial Agents: Mirroring these human insights, AI researchers have theorized that internally motivated exploration can drive creativity and learning in machines. Jürgen Schmidhuber’s work provides a formal framework: agents can be endowed with an intrinsic reward for discovering novel, surprising patterns that improve their world model (e.g. by improving prediction or data compression)researchgate.net. By “maximizing intrinsic reward for the active creation or discovery of novel, surprising patterns,” an artificial agent is encouraged to continually seek new knowledge, much like a curious child or scientistresearchgate.net. Schmidhuber argued that this principle can yield open-ended development and even explain facets of human intelligence like scientific discovery and artresearchgate.net. Crucially, this perspective treats exploration and play not as frills but as fundamental drivers of intelligence. In fact, some cognitive theorists go so far as to say that exploratory behavior is not merely a means to obtain external utility, but an irreducible aspect of agency itselfresearchgate.net. In other words, an intelligent system might need a built-in “urge” to explore and experiment in order to attain the open-ended creativity and intuition that humans display. This stands in contrast to the notion of a purely task-optimized agent that only does what its cost function dictates. The intrinsic motivation view suggests that without a form of playfulness – a freedom to roam beyond immediate goals – an AI’s knowledge and creativity may remain bounded by its fixed objectives.

Curiosity-Driven Reinforcement Learning (CDRL)

One practical implementation of intrinsic motivation in AI is Curiosity-Driven Reinforcement Learning (CDRL). In standard reinforcement learning, an agent learns by maximizing external rewards defined by a task (e.g. points in a game). CDRL augments this by giving the agent an intrinsic reward based on novelty or learning progress – effectively a numerical incentive for curiosity. For example, a landmark approach by Pathak et al. (2017) defined curiosity reward as the prediction error of the agent’s internal forward model (how surprising the outcome of an action was). This way, “curiosity can serve as an intrinsic reward signal to enable the agent to explore its environment and learn skills that might be useful later” even when extrinsic rewards are sparse or absentarxiv.org. Such curiosity-driven agents are motivated to visit states that surprise them (i.e. where their current knowledge is weak), thereby acquiring new information. Empirically, CDRL has yielded impressive results. For instance, agents equipped with a curiosity reward have solved challenging video game levels with sparse external feedback that traditional reward-only agents failed at, by exploring far more efficiently. In one report, adding a curiosity module allowed an agent to reach a sparse goal with “far fewer interactions with the environment” than a non-curious agentarxiv.org. Even in the absence of any extrinsic objective, a pure curiosity agent will roam its world, “pushing the agent to explore more efficiently” and learn a variety of skills simply because learning itself is rewardingarxiv.org. These outcomes echo a key idea: a system driven by intrinsic motivation can invent its own “tasks” to pursue (e.g. reduce prediction error, see something new), which in turn prepares it to handle explicit tasks in the future. In short, CDRL provides a concrete mechanism by which artificial play – unguided, self-motivated exploration – leads to the emergence of useful behaviors and potentially novel strategies that were never directly specified by a programmer.

Model-Based Imagination: Dreamer and MuZero

Beyond intrinsic rewards, another important ingredient for creativity in AI is the ability to form internal models of the world and reason about them. Two state-of-the-art AI agents, Dreamer and MuZero, exemplify how model-based reasoning can enhance exploration and generalization.

Dreamer: Dreamer (Hafner et al., 2019) is a reinforcement learning agent that learns a world model from its sensory experience and then plans within that model – essentially, it can imagine future outcomes without having to try them in reality. “We present Dreamer, a reinforcement learning agent that solves long-horizon tasks from images purely by latent imagination,” Hafner and colleagues writearxiv.org. Dreamer learns an abstract representation of the environment’s dynamics (a latent state space) and uses it to simulate possible action sequences, backpropagating rewards through these simulated trajectories to improve its policy. This allows Dreamer to achieve high data-efficiency and solve complex tasks: for example, by “learning behaviors by propagating analytic gradients of learned state values back through trajectories imagined in the compact state space of a learned world model,” it outperformed prior model-free methods on a suite of visual control tasksarxiv.org. The significance of Dreamer in our context is that it demonstrates an agent using an internal model to support something akin to mental exploration. Even without an explicit curiosity reward, a model-based agent can try out “playful” what-if scenarios in its own mind, analogous to how human scientists conduct thought experiments. Extensions of this idea have combined world models with intrinsic objectives (e.g. rewarding the agent for exploring states that reduce model uncertainty), effectively giving rise to imaginative curiosity – the agent “plays” through its model to discover novel outcomes or knowledge.

MuZero: MuZero (Schrittwieser et al., 2019) is another breakthrough that showcases model-based planning. MuZero succeeded in mastering Go, chess, shogi, and a suite of Atari video games without being given the rules or physics of those environments. It does so by learning its own state transition model and using a tree-search planning algorithm (like Monte Carlo Tree Search) over that learned model. By “combining a tree-based search with a learned model,” MuZero achieved “superhuman performance in a range of challenging and visually complex domains, without any knowledge of their underlying dynamics.”arxiv.org In essence, MuZero figured out how these games work purely from experience, and simultaneously learned to plan winning strategies. The ability to plan with a learned model is a form of reasoning that goes beyond reactive trial-and-error; it lets the agent evaluate hypothetical actions and thus explore more strategically. MuZero’s success underscores that to deal with complex, open-ended tasks, an agent benefits from building its own internal representation of the world’s rules. While MuZero’s training was still driven by an external reward (win the game), its internal model gave it a powerful generalization capability – hinting that an advanced AI scientist might likewise need an internal world-model to simulate experiments and consequences in imagination, a prerequisite for creative hypothesis generation.

Both Dreamer and MuZero illustrate that model-based reasoning can be combined with or enhance intrinsically motivated exploration. An agent with a rich world model can perform “play” in its mind, trying out variations and seeking novel outcomes in simulation before committing to them in reality. This sort of latent-space experimentation could be crucial for developing original theories about the world – much as human scientists use mental models and thought experiments inspired by curiosity.

Open-Ended Exploration and AGI Creativity

The above elements – curiosity-driven learning and model-based imagination – point toward a vision of AGI that actively generates its own goals and knowledge. The question remains: is such open-ended, playful exploration truly necessary for an AI to achieve human-like creativity and scientific intuition, or could an AI reach those heights by purely optimizing a given objective on a fixed dataset? Current thinking in AI research increasingly leans toward the former: that open-ended exploration is a key enabler of general intelligence. A recent position paper argues that we are on the cusp of a paradigm shift from “learning from data” to “learning what data to learn from,” highlighting that in an open-ended real world, an intelligent agent must actively seek out informative experiencesblog.minch.coblog.minch.co. In this view, exploration is essential not just in reinforcement learning games, but in all learning settings. Jiang et al. (2023) coin the term “generalized exploration” and assert that exploration should be treated as a necessary objective to sustain open-ended learning, allowing agents to continually discover and solve new problems – a path toward more general intelligencearxiv.orgarxiv.org. Similarly, researchers at DeepMind suggest that “open-ended exploration provides a more viable, bottom-up path toward general intelligence” than static, task-specific training, and that our focus will shift to designing exploration objectives and data-generation processes for agentsblog.minch.co. In essence, an AGI might need an intrinsic “drive” to pose its own questions and seek novel experiences, which is strongly reminiscent of play.

From a philosophical standpoint, one can argue that true creativity – especially the kind that leads to groundbreaking scientific hypotheses – cannot be fully pre-specified as a reward function. By definition, a creative insight is something unexpected and not strictly derivable from optimizing a known objective. Humans often discover new theories (new “models of reality”) by following hunches, tinkering in the lab, playing with thought experiments, or exploring phenomena with no clear immediate payoff. This free exploration allows for serendipity and the breaking of old assumptions. If we want an AI to achieve similar feats, giving it the freedom and motivation to stray from the beaten path seems important. Indeed, algorithms in evolutionary computation have demonstrated the power of not directly pursuing an objective: novelty search methods, which reward an agent simply for doing something different than before, have “counter-intuitively outperformed objective-driven search” on many deceptive problemslink.springer.com. In one example, a novelty-seeking evolutionary agent found a path through a maze more reliably than an agent explicitly trying to reach the goal, because the latter got stuck in local optima whereas the novelty-driven agent kept exploring alternate routeslink.springer.com. This highlights a paradox: by not single-mindedly optimizing the known goal, the agent can discover better solutions – or entirely new problems to solve. The lesson for creativity is that a degree of playfulness (seeking novelty for its own sake) can breathe diversity into search and yield outcomes that a pure utilitarian approach might never encounter.

On the other hand, it’s worth noting some counterpoints. In certain cases, extremely powerful goal-directed systems have exhibited what humans label “creative” behavior without explicit intrinsic motivation. DeepMind’s AlphaGo, for instance, produced novel strategies in the game of Go (famously, the unconventional Move 37) purely by optimizing the goal of winning games through self-play. Likewise, OpenAI’s multi-agent hide-and-seek simulations led to agents inventing ingenious tool use and strategies – effectively creative problem-solving – driven by competition rather than an explicit curiosity reward. These examples suggest that a sufficiently rich external objective, combined with a complex environment, can induce emergent creativity. However, in both cases the agents were either limited to a narrow domain (games) or relied on an environment that inadvertently rewarded exploration (self-play generates diverse situations). They did not form new scientific models outside their programmed domain. In contrast, an AGI scientist would need to venture into the truly unknown, beyond any predefined game rules or datasets.

Key Insight: The balance of evidence from AI research, cognitive science, and neuroscience supports the view that intrinsic motivation (“play”) is a critical component for open-ended creativity. Curiosity-driven learning endows an agent with an ever-evolving goal: to reduce its uncertainty and find novelty. This ensures the agent keeps expanding its knowledge and can stumble upon new problem formulations. Without such drive, an AI might excel at tasks we specifically train it on, yet fail to ask the unexpected questions or imagine the unconventional experiments that lead to paradigm-shifting insights. In human terms, it would be like a brilliant student who can solve given problems but never pursues a new idea unprompted. To reach scientific intuition – the ability to hypothesize original models of reality – an AGI likely must have something analogous to the child’s instinct to play and the scientist’s curiosity. In summary, while goal-directed optimization alone can yield powerful problem-solving, artificial creativity may require a playful, exploratory spirit at its core, enabling an AGI to go beyond its training and truly create.

I posted this to AMW list today:

Dewayne had two brothers, one lives in D.C. and is fairly successful.  The other, I know less about.  Dewayne sent me a picture of him, his mom, and two brothers at Thanksgiving dinner some years ago; and I probably can't find it (sorry).

Dewayne sold his Fremont house and moved to his childhood home in the Detroit suburbs, about 15 years ago, to care for his mom.  He was the only one of the three brothers willing to do that.  Because of this, he was very well informed about Covid, because he wanted to make sure his mom, in her 90s, didn't catch it, for obvious reasons.  So, he did all the shopping and such, and became a mini-expert on PPE.

His plan was to care for his mom until she passed on, and then had a pal in Fremont that needed a handy person around, and he was planning on moving back to the Bay Area.

He never got that chance.  He did come back, every six to eight months, to have his prostate radiation pellets replenished, and would spend a few days here.  The last time I saw him, January, he seemed much older than I expected him to look (Dewayne was ageless, as all of you know, looks-wise!), and had a bit more trouble getting off the couch; but no cane, and he certainly never complained.  Whatever physical maladies he may have been suffering, he never made it your burden to listen to a list of complaints.

Like many of you, I'm sure, I didn't sleep well last night.  Just sucks, man.   (denial, anger, bargaining, depression, and acceptance..)

He lives on in all of us.  I remember when our friend Dave Wyland passed, I was totally bummed and Dewayne, on the phone, matter-of-fact told me, "Well, when you get to these ages, these sorts of things happen more frequently -- so get used to it!"    Never one to mince words - and to always give you a straight answer, even if you didn't quite want to hear it.

I'll miss him dearly.

---------

Dewayne passed yesterday morning at Kaiser in Fremont.

I met him via Ham Radio and eventually we got to be good buddies and did some projects together. Often roomed at AMW together. He loved Bucky Fuller, seems he met him as a kid, he liked his philosophy. Claimed to be a pilot, but I never knew of him flying a plane. Said he studied psychology, but I don't know if on his own or in college. Long ago, he wrote code for Macintosh; but then got out of the code business and mostly enjoyed Apple's products. He got me invited to Hackers AND AMW. Dragged me in to do the projects in the Marianas. And the Indian reservations. Had him over a few times for Thanksgiving, with some other nerds. Would usually visit when he was in town for his prostate cancer radiation pellet refresh. Death caused by pancreatic cancer; he was gone is just a few days. He had trouble swallowing his entire life but a few years ago he had surgery to fix that. Most recently, he said he had a few blockages in his intestine that were cleared up with surgery. I suspect that's when they discovered the pancreatic cancer. It's very hard to detect that cancer (so far) and when it is detected, it's generally too late. Survival rates are like 12% on a good day.

No friend is 'perfect' and Dewayne kept quite a few things close to his chest. He doled out information as if he were a WW2 officer giving specific information, only, to the French Underground -- or something. Information is power, he used information to, it seems to me, enhance his stature among peers. I could be reading this wrong, tho.

Well, that's it for now, old friend. We'll be in touch via 'wireless'.

​I have a Mathematical Notational Theory of Emotions, like this:

Emotions are, basically, 'fast evaluators' built in deeply to our brains.   Rustling of leaves, probably an animal, get scared!

But some of these are also 'learned', or more specifically, based on 'beliefs' -- some of which may be intrinsic at birth.

For both of these cases, I call these 'beliefs'.

Therefore, each emotion is a function like this:

Emotion = f (x, y, z, ....) where the letters are individual beliefs.

Expanding, it's more like:

Emotion_1 = f(a*x, b* y, c*z , ...)  because the Emotion Fast Evaluator already 'weights' the various 'beliefs'

Therapy is first, noticing under which circumstances an (undesired) emotion arises; and second, noticing which external circumstances seem to 'trigger' said emotion.  And Third, working to disassociate the 'belief' from the subsequent emotion.

Language turned out to be (perhaps to Noam Chomsky's chagrin) the 'perfect' way into human intellect.  Maybe this should not be so surprising.  Our language is much more better than what the bees or whales use.

However, it will NOT be enough for a full "AGI".

It WILL be incredibly useful as "AI Assistants" for the next half a decade, I would say.  There is no question, code slinging is about to undergo tremendous change.  Not to mention Education, Lawyering, Editing, and much more.

But, Commander Data?   Seems to me something more than Large Language Models will be needed.

Recall, now "AGI" seems to me:  Einstein, Newton, Gandhi, MLK, Stephen Hawking, and more --- ALL IN ONE.   the "ALL IN ONE" is the new part.  Heck, GPT-4 easily beats the "Imitation Game" as specified by Turing.  So, the bar has to be raised.

---

Bottom Line: there is FAR too much hype around GPT-4; it's amazingly cool, but is also LESS than MOST people think it is.   

AT THE SAME TIME: OpenAI is not the only folks in the world who know how to make LLMs; you can be SURE there already is some LLM trying to squeeze more $ out of the stock and other markets.  And political regimes around the world are also developing these things.   They are happening *as we speak* and so, in a very real sense, the Age of AI has already begun.

'Retired' is not age-dependent, although does skew older.

What they do is: take up fairly complicated goals -- reach goals, if you will -- which they know they can do if they really work on it, and in the end, there is something nice to show people.

This is clearly the whole raison d'etre of the retrocomputing crowd.

I think another reason is: you get to work on technology personally interesting to you, but not in any way financially motivated. So maybe you learn some web tech, just because, why not?

There is an enormous reservoir of 'retired' talent, working on - well - not really Society Improving Tech - because nobody else has harnessed these folks into a Tsunami of Tech Innovation.

Remember, these folks are working on what they're doing because they want to - they are self-motivated. So any 'Federation' must take that essential fact and use it.

In a sense, self-selecting a task, and then working to achieve it is the basis of all hobbies. You want to develop and demonstrate skill, and most often NOT in the areas you were paid during your career.

"The problems of heuristic programming—of making computers solve really difficult problems—are divided into five main areas: Search, Pattern-Recognition, Learning, Planning, and Induction."

In 2015:

Q: Are machines going to become smarter than human beings and, if so, is that a good thing?

A: “Well, they’ll certainly become faster,” he said. “And there’s so many stories of how things could go bad, but I don’t see any way of taking them seriously because it’s pretty hard to see why anybody would install them on a large scale without a lot of testing.”

Marvin left us Jan 24th 2016

ANN = Artificial Neural Net, BNN = Brain Neural Net, DL = Deep Learning, GWP - Gross Worldwide Product, CNN Convolutional Neural Network (aka "Vision ANN"), OOM = Order Of Magnitude, LLM = Large Language Model, MCTS = Monte Carlo Tree Search, GPU = Graphics Processing Unit, CPU = Central Processing Unit, DQN = Deep Q-Network, VPT = Video Pre-Training, CLIP = Contrastive Language-Image Pretraining, and of course AGI = Artificial GENERAL Intelligence (what we 'used to call' just "AI" but the Capitalists co-opted the term to describe their CNNs)

Silent Cacophony turned 10 today!

This isn’t groundbreaking, but I’ve never seen it -- and I’m not sure why not.

The basic idea is to use some damn fast dual-port SRAM which is mapped into the MPU’s address space; and an external processor is shoveling the data out of the SRAM on the other side, and blasting into a huge (enough) DRAM buffer; eventually you will fill the buffers; but having the last N seconds of high-speed data when some fault occurs is mighty helpful.  In practice, a few variables at highest possible speed is probably better than tons of variables at pretty high speed, because sifting through all that stuff is not easy.  I’m thinking of the Leg Lab PS2 which had maybe 30 possible variables, and a way to look at (it was if I remember) 6 of them at a time in detail in a large portrait-type window.

There are web-based and non-web-based display / manipulation solutions.  Since we’re talking  ‘fairly small’ amounts of data on-screen at any given time, the web-based can get data from an SQL (say) back-end, running locally on the same machine as the web browser.  The variables would be buttons, and one would select some of them to see their graphs.  Since this is not ‘live’ data, but is analysis after-the-fact, no throughput considerations are important except to be snappy at human speed.

Non-Web programs of course do-it-all at once.  Your PySimpleGUI  (or similar) provides the data reading & displaying.  For me, this latter approach would be faster to implement.  It would have the downside that a normal website could easily provide info to others in real time; not needing to capture screen shots or video or other artifacts that would go onto Confluence or something like that.

Doing math between variables to create new variables of course is possible.  Or doing some sort of thresholding, etc. This is a later improvement to the basic functionality.

My best guess as to what he’s saying in “She’s Too Fat”