How can I build a small operating system on an old desktop computer?

Question

This might be in vain, as I know writing an operating system is unbearably complicated (especially by oneself).

• I don't expect to build the next linux, or windows.

• I know it will be horrible, and buggy, and won't work, but that's fine.

I want to write everything myself, in Assembly, C, and (some) C++.

This is a future project, as I'm busy with some other things at the moment and don't have the time immediately, but I figured I would ask it now, so maybe I could get lots of answers to this, and it could build up and be a useful resource for this kind of approach (everything else I have seen involved building off of minix, using an existing bootloader, building it in a virtual booting program thing, etc).

I want to set up one of my older desktops with a monitor, keyboard and mouse, and start working on a blank hard drive.

I want to learn how to write my own bootloader (I've found lots of resources about this, but for completeness, please still add some good ones), my own USB driver (if that's necessary), a CD driver (if that's necessary), etc. Everything, from the ground up.

• How do I put the code onto the computer? Is it best to do it with a floppy disk? Can most computers do it from a USB stick?

• What drivers do I need, and can you suggest any references to building those?

• After the booting sequence--then what? How do I get into protected mode etc.

• How do I manage memory without the help of an operating system? Do I just use whatever addresses I want? No initialization necessary?

• What will I undoubtedly run into that will confuse me?

• How can I make it either a command line O/S, and a graphical one?

• What is a graphical O/S built on? Like, how would I do something like, a command line, with a font, and a picture at the top?

• Where can I read about setting up a multitasking environment? (ie., having two graphical-like command lines running side-by-side).

• How would I set up a sort of windowing system? How do I display graphics on the screen once simple multitasking is set up?

Believe me, I understand that this is a very complicated project, and I probably will never get around to completing it or writing anything on it of any use.

There are lots of other pieces to this I haven't mentioned, if you think of any, feel free to add those too.

Please put one "topic" per answer--for example, USB drivers, and then maybe a list of resources, things to look out for, etc.

Also, please don't suggest building off of another O/S or pre-existing code. I know I will read a lot of pre-existing code (such as the linux kernel, or example resources, existing drivers, etc) but ultimately I want to do all the writing myself. I know I should build off of something else, and there are lots of other questions on SO about that that I can read if I change my mind and go that route. But this one is all about doing the whole thing from scratch.

Any suggestions on how to get it graphical? Different video modes and how to work with them, etc?

Solution 1

First things first. Read, read, read, read, read. You need to have a firm understanding of how the OS works before you can hope to implement your own.

Grab one of Andrew Tanenbaum's books on operating systems. This is the one we used in my OS class in college:

Modern Operating Systems PDF

Modern Operating Systems on Amazon

Despite the ridiculous cover, it's a fantastic read, especially for a textbook. Tanenbaum is really an expert in this area and his explanations of how the OS works underneath the hood are clear and easy to understand. This book is mostly theory, but I believe he also has a book that discusses more of the implementation. I've never read it, though, so I can't comment on it.

That should help you bone up on process management, memory management, filesystems, and everything else your OS kernel needs to do to get it up to a bootable state. From that point on it's basically a matter of writing device drivers for the hardware you need to support, and offering implementations of the C library functions to make kernel calls for things like opening files and devices, reading and writing, passing messages between processes, etc.

Read up on x86 assembly (assuming you are designing this for an x86 machine). That should answer a lot of your questions with regards to moving between processor operating modes.

If you've got any electronics knowledge, it may be easier to start with writing an operating system for an embedded device that has ample documentation, because it will generally be simpler than an x86 PC. I've always wanted to write my own OS as well, and I'm starting with writing a microkernel embedded OS for This Development Board from Digilent. It can run the soft-core MicroBlaze processor from Xilinx, which has very thorough documentation. It's also got some RAM, flash data storage, LEDs, switches, buttons, VGA output, etc. Plenty of stuff to play around with writing simple drivers for.

One of the benefits of an embedded device is also that you may be able to avoid writing a VGA driver for a long time. In my case, the Digilent development board has an onboard UART, so I can effectively use the serial output as my console to get the whole thing up and booting to a command line with minimal fuss.

Just make sure that whatever you choose to target has a readily available and well-tested compiler for it. You do not want to be writing an OS and a compiler at the same time.

Solution 2

http://www.osdev.org/ and http://www.osdever.net/

welcome to the OS Development world.

See also other x86 documentation links in SO's x86 tag wiki: Intel and AMD manuals, compiler / assembler docs, and various guides.

It also suggests using BOCHS or other virtual environment for debugging, since you can single-step your bootloader and examine registers.

Solution 3

I would suggest working, at least at first, on Bochs or some other virtual machine the reason being that you can take it with you wherever you want, it's easier to debug (you can see the exact state of the hardware), and if you need outside help debugging they can use the exact same 'hardware' as you.

The most useful advice I have is to get yourself into a state where you can be running C code as quickly as possible -- i.e. boot up, setup your descriptor tables, and get yourself to a point where it's safe to run compiled C. Most if not all the kernel should be in C if you want to stay sane and keep working on it. Assembly, while required in some places, is tedious and tends to be hard to debug.

Solution 4

At its lowest level the minimum that an operating system needs to be able to do is to drive a system's hardware in some way and somehow load an execute some sort of "user code." If you're going to start with a PC then you need to write code that can be loaded by it from some device or another. Older PCs have a BIOS in firmware which determines how the hardware performs some initialization (at least video, keyboard, and some form of storage or boot loader). (Update October 2017: Newer PCs have EFI or UEFI firmware ... which is largely a pedantic difference; the serve the same purposes for this discussion).

So start by learning the low level details of how to use the BIOS or other firmware, on your target system. That is, learn how to write a program that the BIOS can load and execute. That will eventually morph into your boot loader. Start small. Just get a program that prints: "Hello, Linus" directly from the firmware boot process (on a floppy, or USB thumb drive, would be a good start ... or on a hard drive if you like).

From there I'd recommend writing a very simple serial driver ... update your boot loader to initialize some serial port, and start a download there from. Then it can execute the code it pulls across. From there write a bit of bootstrap that can write to another set of blocks (we haven't implemented a file system yet ... not even partition table parsing; so we'd just deal with raw ranges of blocks on the disk at first).

At that point your boot loader should be able to pull new code across the serial line, dump it into a partition (yes, implement partition table handling of some sort ... whether it conforms to standard PC conventions is up to you at this point), and execute it.

From there you should be able to work on far more sophisticated features. From this base you can write and compile a new "kernel" ... reboot your testbed, and have the new kernel deployed to it.

(Your bootloader should take some signal, such as a BREAK over the serial handshaking lines as a command to skip the download and just boot the existing image; and it should handle some timeout in this way as well).

From there write a very simple terminal layer and command shell? A filesystem? Implement commands to download new executable content other than the kernel (files or objects of some sort). And so on.

Naturally you could have started with a console driver using the PC keyboard and video (the BIOS INT 0x16h and INT 0x10H stuff, respectively, if I recall correctly). However, I'd suggest starting with a serial driver since you can then automate your build/deploy/test cycle from any other existing (functional) system. Since your new OS will start as a cross-compiled project it's going to be essential for you to have a streamlined way of handling that.

I don't know how far you want to take your project. A reasonably impressive goal would be to achieve "self hosting." If you can create a simple assembler/compiler that can allow you to use your new OS to (re-)build, link, and boot into a working version of your new OS ... then you've achieved that goal. (Note that's not a requirement. Many embedded systems are never going to be self-hosting and there's nothing wrong with that).

Solution 5

If you don't mind using hardware virtualisation, there is a course (book + lectures + software) that will take you 'From Nand to Tetris'. You create a full computer system entirely yourself from the (for these purposes atomic, and given) electrical NAND gate, right through to building the OS, a language, and finally coding a simple game on your personal machine.

I think it's a great idea and something I fully intend to get stuck into soon. The book is surprisingly cheap and I believe the course is taught at MIT. I can imagine no greater feeling than having the full, complete knowledge of an entire system you built yourself from the ground up.

Link: http://www1.idc.ac.il/tecs/

View more solutions

Comments

[Max Schmeling]

This is my single greatest goal. I want to write an operating system so badly, just for the fun and learning of it... I just haven't been able to spare the time the past few years. If you're ever looking for someone to help out or just talk through things with you, you can email me at schmeling88 at gmail.com

[Giovanni Galbo]

@Max and @Tom, I got the bug too. I'm actually gearing up to start this time around :)

[Bastien Léonard]

Just don't think about graphics now. Unix never supported graphics directly, and it isn't so bad. :-) But basically there's no easy and portable way to get beyond 800x600 IIRC.

[Ciro Santilli OurBigBook.com]

Voting to close as too broad. Similar to: stackoverflow.com/questions/254149/… || stackoverflow.com/questions/2800741/…

[Carson Myers]

Why? This question is ancient and not generating any activity anymore. Who cares?

[Peter Cordes]

Related Q&As: General bootloader development tips / How to make the kernel for my bootloader? has a working Hello World kernel for an x86 PC that boots in legacy BIOS mode (not UEFI), with a bootloader that switches to 32-bit protected mode and runs a C kernel.

[Carson Myers]

great answer, I already have the linux kernel handy and I'll look into windows 3.1 for sure.

[Carson Myers]

I'll look into Bochs to play around for now, but I eventually want to just be building this on some PC

[Bastien Léonard]

+1, but I prefer Tanenbaum Minix book (which is more aimed toward implementation), and there's really no need to only focus on theory first. He won't write an OS on the first try, anyway. He will first write a bootloader, then switch to protected mode, then handle I/O, and so on.

[Carson Myers]

not exactly what I asked for, but +1 for sending me on a tangent of reading

[Matthew Iselin]

And especially wiki.osdev.org!

[Matthew Iselin]

And you can get paper copies for free through Intel.

[Paul Nathan]

Silbershatz is really quite good, but very high-level.

[amaterasu]

Agreed; However, I'd say its fundamentally essential. Documentation right up there with Modern Operating Systems by Tanenbaum (mentioned earlier)

[Rune Jeppesen]

Marsh is giving you good advice. The low-level details of the x86/BIOS architecture are complex. You'll likely get bogged down on these details, leaving you unable to focus on the bigger issues like memory management and process scheduling. Go for a "cleaner" architecture with a serial port. You'll be happier in the long run.

[user14322501]

Don't forget forums.osdever.net :)

[Kristopher Micinski]

+1 for writing an OS for an embedded chip. At the end of the day you'll feel you made something unique, and it's just massively simpler than writing one for (say) x86, while retaining the same core concepts. Look at TinyOS, for example (while not technically an OS, there are things like scheduler and task modules for it..)

[icktoofay]

I'm aware these are useful resources, but I think it might be more helpful for future readers if you provided some context rather than a set of links without explanation.

[Koray Tugay]

I think this link is dead. Does anyone know any other way to access this?

[user253751]

@KorayTugay nand2tetris.org

[Jim Dennis]

On modern hardware it might be more difficult to get a serial driver working than to write console (video/keyboard) drivers. In the several years since I wrote this, plain old serial connectors have become practically non-existent on modern desktop and laptops. Some embedded and "hobbyist" hardware still has old-fashioned RS-232C interfaces without the need to go through USB emulation. For examle: mpl.ch/t2600.html seems to have it.

[Kamila Szewczyk]

Their code is broken. But tutorial itself is great.

[Peter Cordes]

If it works in BOCHS, it will usually work on a real PC, modulo differences in their BIOS. Most of your bugs will probably be things that affect BOCHS and real hardware, but the difference is that when your code gets stuck in an infinite loop in BOCHS, you can use BOCHS's built-in debugger to find out where, and single step to find out how/why. On real hardware, all you could do is add debug-prints. Using a debugger will save hours and days of head-scratching.

[user253751]

@icktoofay The question is too broad to be answered properly, I think.