Learning by doing: writing a mini OS for a Z80 based system - looking for HOWTOs and advices

Software
1

Hi,

as a project to improve my skills, I would like to try and write a small operating system for a Z80 based computer (ZX Spectrum). Because I have no formal computer science education (I am a kind of self-taught bedroom coder, mostly demoscene oriented), I am missing quite a few concepts in this field which I feel I should learn. I would be really thankful if you have any advice and/or documentation I should go through before I start to write anything.

As for my idea, I am thinking rather small. Here is what I am NOT trying to achieve:

  • multitasking
  • GUI
  • compatibility with any existing software

I am interested in some simple text-based console for running programs, possibly with arguments, kind of CP/M style. I would like to be able to switch between the OS and program(s) running - when I switch back to the OS, the program will be stopped in the background and would continue to run once switched back to it (I hope this make any sense). All the I/O should be implemented as a part of the OS and programs should use system calls to use it.

The platform would be ZX Spectrum 128K, here is a brief description:

  • there is always ROM paged-in at #0000 - #3FFF
  • there is always screen memory paged-in at #4000 - #5AFF
  • rest of the RAM is free to use
  • next memory can be paged-in in 16kiB blocks at #C000 #FFFF

For a storage, at the beginning I could use the ROM (so basically a TAPE) as it is fast in an emulator, later I would switch to use the divIDE (an IDE interface). In both cases I don’t have to care about implementing a filesystem (I would just call the EsxDOS services in divIDE).

At this point I am not 100% sure where to even start - should I think about some basic features of the kernel now or should I start with the text console and build the things around that? Too many questions at the moment.

I did some search around this forum (and the internets) and now going through the Bootstrappable (Retro) OS? also WHEATSYSTEM and couple of others. But I feel like I am missing the basics so what ever I write now would very likely end up in a mess.

Anyway, if you feel after reading all this that I should give up right now, it is fine to say it. But I am really curious.

2 Likes
2

Make sure you have good power supply. so you can handle what ever i/o you want to add or use.
Knowing the hardware works makes fewer software issues. The C64 was kind of neat in that
I/O was handled mostly the same for all devices as serial I/O. Disk I/O often takes more space
than you expect for I/O buffers. CP/M used a tiny 128 byte sector, more modern devices expect
512 bytes/sector.

1 Like
3

Should you give up right now? By any means, no!
However, doing it in higher level language, like C, may be a gould advice for the beginning. (You can always change parts of this to assembler, if you mind performance, or for pride, or {insert here}.)

Once, I actually did something much like this, starting from a similar background. Only, in my case, it was for a superficially UNIX alike system that should run in a browser. The latter was then (in 2003) the tricky part, since this was still the browser war and this must support anything from Netscape Navigator 4 and Internet Explorer 4 upwards, and even that new-fangled thing called Firefox (then still in its 0.x days. :slight_smile: ) Anyways, my background was/is similar to yours, with about as much insight into UNIX, etc, as you get when dealing with a web server. I even avoided reading up on this, rather, having a try at what I could come up with on my own.

So, in my interpretation, there are a few boundaries to what may considered a OS: The user facing I/O, we receive numeric keyboard codes and in turn write numeric character codes to the console. Then, we want to access and manage files (this was cheap in my case, since these were just stored in variables and the browser was lacking any real storage access) and running binaries (in my case, these were JS scripts, which were called by some glue logic). In the middle of this, you have a file system (I went for the conventional inode and file pattern, where the inode contains the descriptive attributes, and the file the payload), process management and a shell as the magic glue between these components. (I went with the UNIX way of having the current working directory managed by the shell, rather than, say, in the core system, which simplifies things.) Process management had to be cooperative, since I was sitting on top of a single threaded system that was also considered to be the “native” layer.

My advice would be to manage these “boundaries” first, console I/O and files, and a start mechanism for any processes. Also, you may want to define what goes with a process. E.g., an environment object of sorts is nice to have, and of course, a convention for an ARGV (arguments vector comprising any arguments handed to that process) and code to access this from inside a process. For scheduling, some sort of parent-child relationship is probably a requirement. You’ll probably have at least some sort of top-process, and children, which do their jobs and yield their results and status their parent. In order to chain this up, a process should know about its children and should have a way to yield control and status to its parent.

A good way to start may be to have a console which can receive and parse a filename and type that file, if there’s any. (This already provides you with code for the essential “boundaries”.) Then, there’s still time to invest into more glue logic and make things more interactive, replace the code for typing back the file contents by arbitrary programs/processes, make them replaceable/callable by filename, etc. From this perspective, the tricky and vital “middle part” (scheduling, process management, a more comfortable shell, a full-fledged file system, extending the console to support styles/character attributes, etc) becomes just some more of the glue logic. Iterative refinement is your friend.

Speaking of the console and interactivity, mind that it may be useful to have two modes, cooked and raw, where the former asks for word-like chunks terminated by an ENTER code and allows for line editing (mind that an input line has two ends, when it comes to editing), and the latter is about unprocessed character codes, as may be required for a game or an editor. (And a process should be somehow able to somehow communicate to the console what it is about.)

In a higher language and in a convenient development environment, it can be done actually quite quickly, as, much like when writing a novel, much develops consequentially from what you already wrote and also “crystalizes” while you write what comes just before. (For which it is also advisable to have some crude outlines of a plan, right from the beginning.) In my case, I had the essence of what was needed for a console already around and the “rest” was about two weeks, in which I, admittedly, didn’t do much else. (For an excuse, I was curing a cold.)

If you want to visit my youthful sins (this is now more than 20 years ago), they can be seen here: https://masswerk.at/jsuix/ (feel free to have a look at the source, it’s dated vanilla JS w/o dependencies) and a crude outline/description can be found here: https://masswerk.at/jsuix/jsuix-documentation.txt. (This stops with “to be continued”.) Finally a compilation of the man entries is here: https://masswerk.at/jsuix/man.txt

3 Likes
4

Go to the dark side … Java anything.
BCPL has z80 version, for the light side.
Looking at the memory map, you really don’t have ram for programs
A flat 48Kb segment is needed in my view thinking of typical cp/m programs.
Toying with homebrew cpu, I had 32K words for programs
and 32K words ram for the OS. 8K upper memory reserved
for rom and i/o. The fpga card I was using was too flakey
to get a real os written.

1 Like
5

Seconding @NoLand’s advice here, step one is don’t give up.

At one extreme, you’d do lots of study and lots of design and then start implementing some low level facilities. But at another extreme, you’d start with a minimum viable product: perhaps a command line which understands a trivial command like ‘echo’. And build up from there.

Sounds like a good starting point to me. But

is starting to sound like a multi-process sort of thing. It was a little while in the development of unix shells that task management (edit: job control) came in, IIRC. Early on, you can start a program, and you can abort a program, and that’s it. But that’s a lot.

My advice would be to make a start, don’t be too ambitious, keep building onto something which works.

1 Like
6

Yeah, my plan is to use z88dk.org (so, C) for (at least) the prototyping. I am not aiming at the optimizations right now, I need something up and running first.

You are right, I should start otherwise I will be overthinking everything.

I will take a look, thanks!

Yes, by “programs” I mean something very small, read a text string, print a string (basically an ECHO as @EdS mentioned) or beep. The whole project is strictly for learning purposes and not for real general use. If things will go well, I may move to the Spectrum +3 in the future, I can page out the ROM there to get another 16kiB RAM space. But that is for another story.

I appreciate all your replies, I will start and will post my progress once a while.

1 Like
7

Well, if you plan on building an OS for the ZX Spectrum, I’d suggest first getting used to writing compact code and maybe start with a challenge first. I’m happy to suggest one if you feel it’s something you’d also like.

Try to write a tiny memory and bit viewer that could look like this:

Mockup1

You would have a memory listing as suggested, a bit viewer just above it and a command line below.
The command line would accept:

  • addresses: (ex: type 8000 and press enter) which would move the view to that address;
  • modify commands: (ex1: type “8000, FF”) and that would effectively “POKE” FF at 8000)

You could use some keys to scroll memory pointer up/down/left/right. In the illustration, the pointer is at 8000

The top panel represents the bits of all bytes represented in the viewer below as suggested by this other ilustration:

Mockup2

The program should be NO MORE than 2048 bytes in total, it should be compiled at 16384 and ran with RANDOMIZE USR 16384
As you perfectly figured, that’s in the video memory, but we don’t care since the bit panel we’re rendering will represent its data in the color page starting at 22528 and you’d be using the same “INK” and “PAPER” color to mask the actual code.

You would need to set up:

  • getting typed input from the user;
  • parse input;
  • print text (I think there’s a subroutine somewhere in the ROM for that);
  • display the memory bytes from the given address on the middle part of the screen;
  • display the bit state on the top color page of the video memory.

Bonus points if you make the code relocable (i.e. don’t use JP, use JR, don’t use fixed global addresses unless calling stuff from ROM).

It’s not much in the direction of an OS but I feel it would give a lot of useful information, you can complete it and then you can reuse code for other projects. :wink:

2 Likes
8

Also, keep in mind that UNIX (allegedly*) started as some glue logic to start a game (Space Travel) on a machine, which was about as limited as the venerable ZX Spectrum. (There was no real file system, no shell, no anything.) – Even big things start small…

*) not necessarily 100% historically accurate.

A path to this may look like this:

  • make a console that can collect and write back what you type (after hitting RETURN) – always a great starting point
  • make it so that you can identify a certain word and then write back a certain thing (e.g., an “info” command)
  • make it so that there can be multiple things (hey, files!)
  • make it so that these things can be runnable code
    (any stored states are already some kind of process information)
  • make it so that these things can receive input
  • make it so that these things can read/request other things (hey, already an entire OS!)
  • if still at it, add logic to organize things.
3 Likes
9

To get those missing concepts you might want to pick up a copy of Doug Comer’s XINU book. This is the one I mean: https://www.amazon.com/Operating-System-Design-XINU-Approach/dp/0136375391 (I don’t know why that particular one is so expensive; I got my used-like-new copy for around $30 - maybe check eBay?).

Make sure it’s the first version for the LSI-11 and not the later PC or MAC version. Even though XINU does have multi-tasking and is kind of visually similar to UNIX, it’s better suited to run on the limited hardware of our retro-systems.

An alternative would be to get an early edition of Tanenbaum’s “Operating Systems: Design and Implementation.” The first version was aimed at the original IBM PC, and so is only shooting a tiny bit over the head of your target system. :slight_smile: That book is far more conceptual (in spite of coming with loads of source code to study) and is not so much of a “how to do it yourself” manual.

Both books are great though!

1 Like
10

Hey, thank you for the challenge, I like that one actually! I use ZX Spectrum because I do very well know that computer so it is easier for me. I will take a look at the memory viewer idea at some point.

@NoLand I actually did very similar list and working on the text “console” now:

  • get an input, quit
  • get an input and write it back, quit
  • parse the input and quit if the string is “exit”
  • list prepared strings in memory (ls)
  • rename the strings

@Paganini

Thanks for the books, I will check those.

2 Likes
11

I recommend the book The 8080/Z-80 Assembly Language: Techniques for Improved Programming by Alan Miller published 1981.

The book presents in detail an 8080 and a Z80 system monitor with fully commented, clear source code. This monitor can be the starting point for something more ambitious like an operating system.

2 Likes
12

One could get a floppy drive back then. 64K ram well matched the 360K 5 1/4 inch floppy
and small sector size 128 or 256 bytes per sector. Using todays media may require a
bank switched cpu due to the large sector size and number for blocks per disc and other
OS demands.