Umeet 2001, opening lecture

This year I got the honour of doing the kick-off lecture for Umeet, I will be talking about my favourite project, the Linux kernel. Like a traditional keynote, this talk will not be as technical as my usual lectures ... but I will say some technical things once in a while ;)

Kernel 2.4, the last year

january 2001: kernel 2.4.0 released
- about half a year after 2.4.0-test1
- for the first time, people really start testing 2.4
- lots of bugs get discovered and fixed
march 2001: kernel 2.5 summit
- an event in San Jose, USA, where kernel hackers meet each other to discuss the future of the kernel
- RedHat hackers debug the 2.4 kernel the week before the conference
- many interesting things got discussed, much alcohol and coffee consumed
july 2001: Linus gets busy
- bugfixes from various people, including subsystem maintainers and Alan get dropped
- Alan Cox does have time to integrate the patches
- Alan merges S390, PPC, SPARC, etc.. updates
- Alan merges User Mode Linux
- Alan merges ext3 and intermezzo filesystems
- as a result, Alan's 2.4 kernel patches grow into a full-blown kernel fork
- most distributions start using Alan's kernel
- due to a lack of flow control, some people give up submitting patches while others keep flooding Linus ... the situation never recovers
august 2001: Alan's kernel stable
- Alan's kernel stabilises to the point where it runs heavy stress tests on complex server hardware
- Linus still misses some important bugfixes and adds new experimental code
- Alan merges
- the kernels of Linus and Alan move further and further apart, by this point the VM is basically unmergeable
september 2001: new VM
- Linus gives up on the old VM ("unfixable") which is fixed in Alan's tree, integrates VM by Andrea
- Linus his kernel gets unstable for a while again
- porting bugfixes from Alan's kernel to Linus his kernel gets even harder
november 2001: official 2.4 stable
- Linus stabilises the 2.4 kernel, fixes the last important bugs
- the 2.5 kernel gets split
- Marcelo Tosatti becomes the new maintainer of the 2.4 kernel
- a lot of bugfixes and updates are still waiting to be integrated into 2.4
- Marcelo is buried alive in patches

Things to happen next year

A number of interesting projects are going on with the kernel, I don't know which ones will be merged into the kernel, I'll just present a few random ones:

User Mode Linux: new "supported architecture", allows Linux to run on ... Linux
- kernel debugging
- sandboxing of users
- testing new kernel versions
- ccClusters ???
- User Mode Linux
Inter-Mezzo: replicated network filesystem
- replicates the data over multiple machines
- allows for disconnected operation
- good for laptop/desktop data replication, web server cluster, ...
- also useful as backup/raid device ;)
Linux Security Modules
- plug-in scheme for loadable security modules
- can be used as a framework to load other security systems on a running system
Dprobes
- dynamic debugger which works for both kernel and userland, can be used to debug anything running on the system
- project by IBM India, originally for OS/2
- more flexible than most other debugger projects
ext2/3/4 developments
- people are still working on various advanced features for ext2
- hashed directory layout
- extents
- larger block sizes
- tail merging
drbd
- distributed r*mumble* block device
- used to replicate data across 2 machines
- useful where shared fibrechannel disks are too expensive, but data needs to be accessible intact when one server crashes
- used for filesystems, databases, etc...
OpenGFS
- Sistina took their GFS commercial, so a free version was forked
- GFS is the Global File System
- allows multiple computers to share the same physical disks and the same filesystem, data, etc...
- useful where the data needs to be shared between multiple servers
- higher performance than eg. NFS, better data sharing than eg. intermezzo
- needs special hardware for best performance
memory management (on request)
- after the "natural selection" thread of last weekend, I'm now convinced that it's better to develop memory management separate from the kernel
- inspiration from various systems, Linux, FreeBSD, NetBSD, Solaris, SysV, etc... (many publications are available)
- light-weight implementation of what many other systems do
- page scanning per physical page, with mappings from physical page to page tables using that page
- large code cleanup
- cleanup, hopefully simplification of SMP locking
- patches should be available regularly, I hope ;)
CPU scheduler
- current scheduler has problems when the number of runnable processes is large, a common situation on machines with lots of CPUs
- faster process selection
- per-CPU data structures, so most SMP locking can be avoided
- various designs are being tried, IBM, Alan Cox, David Libenze, ...

Conclusion

Since Linux evolves naturally, without much planning, there is know way what will happen in the future. The only thing we know is that some of the nice projects will get integrated into the kernel, but also that everybody has an influence on what goes into the kernel ... the future of the Linux kernel is up to you.