Cat Notes

Browserify starts at the entry point files that you give it and searches for any require() calls it finds using static analysis of the source code's abstract syntax tree.

For every require() call with a string in it, browserify resolves those module strings to file paths and then searches those file paths for require() calls recursively until the entire dependency graph is visited.

Each file is concatenated into a single javascript file with a minimal require() definition that maps the statically-resolved names to internal IDs.

Founded in 1926 by James McKinsey (accounting professor), who basically applied accounting principles to management (the guy invented budgeting) and sold his firm has doctors for companies with problems. After his death, his two partners split up the firm and Marvin Bower (lawyer) took established the practice of hiring young intelligent MBA grads, the up or out principle, and the idea of giving management advice to the most elite of companies (instead of being a bad company doctor). 

The company profited from advising on military industry during WWII, disseminating the M-shaped organizational hierarchy during the M&A boom of the 40s, assisting US company expansion into Europe during the 40s and 50s, and pioneering “strategic consulting” for professional managers in the 50s and 60s. Marvin Bower stepped down in 1967. It was characterized by a lot of pretension (e.g. a firm with members, not a company with employees, consultants didn’t work a job, they played a role, McKinsey did not negotiate with clients, it made arrangements, it did not have customers, it had clients, etc.). In some ways, they defined the language for consulting as it is today. 

During the 60s, McKinsey faced significant competition from BCG and other rising consulting firms that sold “products” instead of just the raw intellect of its consultants. Examples of such products were BCG’s growth share matrix and their experience curve. McKinsey responded to competition by developing a comprehensive knowledge management system of experts and papers. This began as a little red book of phone numbers and a culture of everyone gaining and chipping in a particular vertical expertise (functional or industry). This gradually became a database of papers, case studies, and information for posterity. Ron Daniel (MD 1976-88) pioneered the knowledge management efforts. 

Heap memory safety

All local pointers point to the SIP’s heap

Pointers only come from SIP’s trusted memory allocator

Pointers are strongly typed

Exchange heap memory safety

EH pointers obey linear type discipline

No double access

Strongly typed

Channel contract agreement

Need to verify that both ends of a channel agree on the channel contract

Kernel ABI agreement

Instruction safety: can’t use privileged instructions inappropriately.

Channel contracts checked for unhanded messages

Channel contracts need all cycles in contract states to contain at least one receive and one send action.

Basically avoid A sending infinite messages to B without waiting for response. This can overflow buffer.

At SIP install time, verifier checks the byte code (byte code in MSIL).

Alternatives: message copying or kernel buffering. Performance not a convincing reason to use exchange heaps—micro benchmarks != application performance.

Kernel responsible for managing the exchange heap’s memory:

It GC’s the exchange heap to eliminate objects held by exited SIPs. 

Singularity also prevents processes from simultaneously accessing objects in shared memory. 

Exchange heap explicitly designed for IPC. 

Linear type discipline for exchange heap objects. 

Each process can have at most one pointer to an exchange heap object at a time. When a process sends a message, the type of the send “system call” forces the sending process to lose that sole pointer to the message. 

Each exchange heap object is accessible to at most one process at a time. 

Sing# is a garbage collected language

Each process has page-disjoint heap. Why?

No shared memory (easy verification)

Simple accounting: memory size == page count

Simple recovery after process exit. Just reclaim all pages for garbage collect.

Simplify experiments with address space designs. what?

Evaluating Hardware Isolation Levels

You can use Singularity to evaluate levels of hardware isolation:

No runtime checks (bounds checks, null pointer checks).

Physical memory: default Singularity; segmentation no paging.

Add 4KB Pages: turn on paging.

Add separate domain: separate duplicate page table for application process, context switches require register %lcr3.

Add ring 3: give application process user privilege. Context switches require protected control transfer (syscall/interrupts).

Software Isolated Processes

Processes run in same address space as kernel, isolated by software only.

Why not use hardware? Performance (kernel crossings are expensive).

Don’t need hardware virtual memory if you have a memory-safe language. Hardware VM needs TLB. TLB has extra cost. 

Want reliability, robustness.

How?

Memory-safe programming language Sing# (no pointer casting).

IPC by contract-based channels—specifications verified by compiler.

Ensures that “every process has explicit code to handle every possible message”.

Drawback: uses advanced compiler, difficult for integration.

Manifest-based programs: type checking in process creation. Manifest says what ABI a program needs, what IPC interfaces are required, dependencies on other processes.

The exokernel’s sole function is to allocate, deallocate, and multiplex physical resources in a secure way: physical memory, CPI, disk memory, DMA channels, I/O devices, translation look-aside buffer, addressing context identifiers, and interrupt / trap events.

Security enforced by associated every resource usage or binding point with a guard that checks privileges.

What is the minimum functionality the kernel needs to provide in order for this primitive to be implemented in application space?  

Address space

OS must support bootstrapping page tables, alloc physical memory, modification of mapping hardware (TLB) and exception propagation.

Simple bootstrapping: small number of guaranteed mappings to map PT and exception handling code.

Physical memory allocation should support requests for given page number.

Privileged instructions can be wrapped in sys calls, and writing to privileged state is associated with access checks.

Exception propagation: save scratch registers in application space then jump to app-specified PC address.

Process

Need: exception program counters, assoc address space, prologue and epilogue code to be called when time slice is initiated and expires. Let the application do the context switching.

Inter Process Communication (IPC)

Transfer a PC from one protection domain to a value in another, donating the current time-slice, installing the called domain’s exception context, and indication of which process initiated the call.

Problem with micro-kernels

Allow replacing device drivers and high level servers, BUT this can only be done by trusted applications.

They still try to provide a virtual machine to applications.

Rigid interface is rudimentary compared to monolithic counterparts. They basically cut a shit ton of necessary features in order to be simple.

Basically, they try to make an OS small, but do it in the wrong way.

“The standard defi􏰊nition of an operating system is software that securely multiplexes and abstracts phys􏰋ical resources􏰀 We believe that this defi􏰊nition􏰁, especially its view of the OS as an abstractor of hardware􏰁 is crippling and wrong􏰀 The basic intuition behind our arguments is that no OS abstraction can 􏰊fit all applications􏰀.”

Ammunition:

Secure bindings

Visible resource revocation

Abort protocol

Desired result

Efficient due to low level primitives

Low level secure multiplexing of hardware resources, low overhead

Traditional abstractions efficiently implemented at application level.

Fundamental belief: use application level resource management. Exokernel securely multiplexes hardware resources. Library operating systems are better at implementing higher level abstractions. 

Problem: traditional OSes limit performance, flexibility and functionality because the interfaces are fixed. Exokernel provides these different things: 

Application level management of physical resources. 

Kernel exports hardware resources to library OS. 

Library OS implements system objects and policies. 

Separate resource protection from management. 

VM and IPC can be implemented within application level library. 

CPU, bus, memory, file system, devices. 

DMA (direct memory access between memory and I/O device)

I/O bus, system bus. 

System bus enables CPU communication with memory. Connects to I/O bus through bridge. I/O bus manages devices and file system I/O, communicates to system bus through bridge. 

System bus can also contain things like frame buffers. 

Hydra OS. Capability approach. 

Capability-based mechanisms for resource access. 

Resource managers as coarse-grained objects to reduce border-crossing overhead. 

Mach. Microkernel approach. 

Extensibility, portability. Mediocre performance. 

SPIN. 

Colocate kernel and extensions. Avoid border crossing. 

Compiler enforce modularity (strongly typed programming language). 

Can’t cast pointers around. 

Logical protection domains, doesn’t rely on hardware. 

Dynamic call binding. ==> flexibility. 

Each app has own address space. Kernel exports mechanisms for accessing hardware resources: threads, iter-process communication, and address spaces. System services are implemented in own hardware address space. No difference with user applications. Only the microkernel has privileged mode--it exports access to hardware resources.

Benefits: extensibility of OS resources, option to customize.