Bootstrapping EMAS-3 for S/370-XA (and a new Edinburgh IMP compiler)

Hi all,

This is a long post, so TL;DR: I’ve rebuilt enough of Edinburgh’s EMAS3 Operating System to get the loader, supervisor, Director, spooler and mailer running on Hercules. It’s still not fully functional due to system processes failed to start. Project files and instructions are avaialble at:

LLM is assisted me in coding but not writing this post. I have carefully reviewed LLM generated code and make sure there is no slop, I also discussed how LLM assisted me over the process in the post.

I’m a Computer Systems PhD student at the University of Edinburgh, where I also did my undergrad. I’ve always been interested in older systems, both out of curiosity and for ideas I might borrow for my own work.

I learned about EMAS and Edinburgh IMP at the Informatics 60 Years of History event. There’s also an earlier thread about EMAS here. I wanted to try it myself, but there wasn’t a straightforward way to run it. It felt like a missed opportunity: here was an operating system developed at my own university that I knew almost nothing about. I’ve wanted to get it running ever since.

The IMP compilers I found weren’t in a state where I could use them to bootstrap EMAS. During my third year of undergrad, the wonderful Compiling Techniques course, introduced me to xDSL. It’s a compiler framework written in Python, also developed here at Edinburgh. Having that available made writing an IMP compiler seem much more achievable. I started with a RISC-V target, wrote lexers and parsers for IMP77 and IMP80, and got Hello World running. Then other work took over and I put it aside.

Recently, feeling burnt out with mobile wireless research, I picked this up again as a change of subject. I discovered that xDSL now had an LLVM backend. That meant I could lower IMP to LLVM IR and let LLVM deal with instruction selection, register allocation and machine-code generation. It removed a big chunk of the work that had been left in my original attempt.

LLM coding tools had also improved enough to take on a lot of the work. I brought back my old lexer and parser and got a new backend and runtime library working well enough to run simple test cases. From there, I could explain the high-level design to LLM coding agent and let it iterate through the many corner cases using old IMP tests.

One design choice was to keep INTEGER at 32 bits and LONG INTEGER at 64 bits, while making ADDR return a pointer-sized LONG INTEGER on 64-bit targets. This is violating the IMP80 spec, but it follows LP64 data model used by other languages. For code that needs different declarations on different machines, I added overrides for ordinary IMP constants used in constant %if statements. The source can still declare a default for older compilers to use.

I also wanted IMP and C code to be able to call each other. The ABI uses the target’s C calling convention, with an imp_ prefix for ordinary external IMP symbols and explicit aliases for calling existing C functions. IMP strings keep their length byte, and generic NAME and array parameters use documented descriptors that C code can construct.

The result is ximplang, now available at:

There was some discussion here about bootstrapping EMAS a few years ago. My approach is to use ximplang to compile the historical IMP compiler into a native Linux executable. That executable retains the original IBM code generator and compiles the EMAS sources into IBM object files.

Unforunately LLM is not so good at dealing with IMP code, persubalely due to lack of training material. I wrote/fixed most of IMP code by hand, but it’s helpful to have LLM on help me fixing syntax issues based on compiler errors given that I’m not that familair with the language.

The historical compiler needed some care to run on a modern host. First I had to write a lot of supporting routing and wrappers to get the compiler running on Unix as it’s origitally designed to run on EMAS.

There a a couple of missing code pieces like opcode list files for the compiler and some wrapper files and I was managed to restore them based on their interfaces.

The compiler stores pointers in INTEGER, so it won’t work on 64bit target. I quickly fixed that, but then I found it also uses INTEGERs for IBM addresses and packed fields. I had to separate the host pointers, which need to be wider, from values that must stay 32-bit. Record overlays also assume big-endian byte order in places: on a little-endian host. To deal with endian and word size issues, I just try to get everything working on a PowerPC 32bit BigE ndian host first, and then gradually fix x86_64 support by comparing the output.

Floating-point constants needed similar attention. The host uses IEEE floating point, while the IBM target expects hexadecimal floating point, so copying the host’s bytes into an object file gives the wrong value. I also fixed the handling of 128-bit constants and preserved the IBM bit patterns of raw constants.

On the EMAS side, I had to get the disk layout, device assignments and console handling to agree with what the system expected. Filesystem initialisation needed to leave enough room for the system images, and Director’s startup checks needed two filesystems configured. A lot of Python scripts are written to deal binary formats and provisioning the disk.

The old 3270 console code needed a compatibility adjustment for screen addressing through Hercules.

There are far too many details I have dealt with to list here, but the end result is that EMAS now boots on Hercules. It’s not a complete system yet, but it’s a start. Mailer process failed to start for some unknown reason.

I’m too excited to share my progress, so I just asked LLM to go over my shell history and make a set of makefiles and scripts out of it so the process can be reproduced.

All the code and instructions are available at EMAS3-Revived. I hope it will be useful to others who want to explore EMAS. I’m also happy to answer questions about the project and help others get it running.

I think there are still a few people from the EMAS days around here. If you used it or worked on it, I’d love to hear from you whether it’s technical details or just memories of what it was like to use. I’m still working towards usable terminal session, and any pointers would be very welcome

4 Likes

Cc: @gtoal and @EdS :slight_smile:

1 Like

What a megapost - great!

I was a mere user (as a visitor from another place) in the very early 80s but it’s great to see posts about EMAS. So much work in universities lost out to other places and ultimately Unix.

-Gordon

2 Likes

Good to see all that Scottish work up and running again.

Impressive amount of work! It does make the effort of having located everything for the archives feel worthwhile, to see a current Edinburgh student take this on. It’s going to take a little time to go through all of your code and reproduce your environment to try it out. (Time I don’t have personally right now as I’m right in the middle of working on our portable Imp77 to C compiler). We did get the original imp80 for intel binaries to run again recently though we haven’t posted the steps to do that, so they’ll be useful in validating your imp80 reimplementation. Hopefully Bob Eager (who is working on the 2900 EMAS restoration, but who also knows EMAS for IBM inside-out) will have a chance to run up your Hercules environment and pass on his advice about the steps to get the subsystem up. I take it the intention of your implementation is to use as a bootstrap for the IBM version of EMAS (which we only have in source form, no or few saved binaries) to then get it to compile itself natively?

Thank you for doing all of this and for putting it online, your enthusiasm is very much appreciated and we’ll really look forward to seeing it all run. (Maybe when you get it going with the EMAS logins you could host an interactive demo service online somewhere?)

I’ve passed on your announcement to our mailing list for imp compilers (which you should probably join - email Bob Eager to get added) but I’ll let you make the post to the edinburgh-computer-history group on Facebook to let the wider community know.

Graham

2 Likes

Maybe you could collect all the IMP material you can find into one place (the Wikipedia page seems to have plenty) and first tell one AI to compile a synthesis, and then take another AI to refine that, continue for a few rounds. Can’t promise it will (a) be correct (b) helpful for synthesizing IMP code, but worth a try?

1 Like

I would think imp to BCPL would be more interesting:). What does the current imp compile to?

I managed to get your ximplang compiler running on my x86 machine (Ubuntu running under WSL Windows11).
I’m going to compare the results of ximplang against some test programs/issues raised against my IMP2026 compiler version (available at GitHub - siliconsam/imp2026: Updated IMP77 (supercedes imp2024 version) · GitHub).

1 Like

As one of the worlds last active BCPL programmers, I think it would be terrible.

Now Imp77 to the same bytecode my BCPL systems compile to (CINTCODE) might be far more interesting… To me, anyway.

-Gordon

1 Like

Thanks a lot for your effort!

Re environment setup, I tried to make it as straightforward as possible, to the point that the whole build process should be essentially a two-liner. I’ve also set up GitHub Actions for continuous integration, so a Hercules disk image is built automatically on every commit. :slight_smile: If setting it up locally is still too much trouble, modifying the code and pushing it to GitHub and see it run would also be an option.

Re the ximplang compiler, yes, the original intention was to use it as a bootstrap compiler to build the original IMP80 compiler. However, the original compiler is such a complex piece of software that it has effectively become a stress test for my implementation. So ximplang end up becoming a reasonably complete IMP80 compiler at the moment. I also did IMP77 parser/lexer and it can pass all old tests, tho I guess IMP77 support is not as complete vs IMP80. Leveraging LLVM, it can compile to a lot of platforms, I personally tried amd64, MIPS and powerpc32 Linux target, but expanding to other arch & OS should be trival.

I also posted it to the Edinburgh computer history group, thanks for the pointer.

1 Like

The ximplang itself lower everything to LLVM IR, and then LLVM can emit machine code for a lot of platforms, x86, arm, mips, riscv…

I want the 1970’s programing course, for computers that took up a wall just to run FORTRAN IV. I don’t have a x86, arm, mips, riscv … but 1970’s cutting edge tech, that can self compile.I want the 60’s programing course to see how ALGOL was compiled. It is the forgotten tech that interests me, not the latest load/store architecture.

.

1 Like

Maybe fast forward to 1980s for MIPS xD

It should be possible to write a lowering backend for stack machine, however modern SSA based IR is a little bit overkilling for this purpose I guess.

Another route is to run old compiler in emulated EMAS, that would be easier.

1 Like

The 1980’s computing was no MIPS, but a 16K IBM 1130 for me.I took electronics engineering
and the computer science dept, would not let us use the VAX as we had our own computers.
A PDP-8 loading 4K FOCAL off paper tape, took good half hour out of hour lab session.

The main reason I was looking a E-IMP around 2000 was that it ran on small machines.
I was using a FPGA board for home brew development at the time and had 256Kb of static ram.
I now have 128K words on this machine,(CPLD no FPGA) and no virtual memory or bytes.
BCPL friendly. :slight_smile:
A major constraint compared to what modern software expects.

1 Like

The IMP compilers developed by Peter Robertson and other Edinburgh alumni generated an intermediate code called iCode from the pass1 stage of an IMP compiler. The pass2 of the compiler converted the iCode to the machine code of the target machine. IMP2026 follows the same style of operation. Most of the iCode instructions are documented under the siliconsam github site.
It might be an interesting project to convert the IMP iCode to the BCPL intermediate code.

1 Like

Maybe time to blow the dust of this again…

For context; I was one of the school kids who used to visit Moray House on a Thursday night (I think it was Thursday night) and pester PSR, etc. and play with Mouses…

-Gordon

Whilst working at Ferranti Electronics, I met Peter Robertson when Lattice Logic were converting their CMOS design suite to generate IC designs for the Ferranti ULAs.
I liaised with Lattice whilst using IMP-77 to code up the generation of the physical layout of the logic cells mapped onto the Ferranti ULA cells.
Still have my copy of the Lattice Logic IMP-77 manual (not the Edinburgh manual).
Loved IMP-77 as it was similar to ALGOL-60 which I used at ICL.

2 Likes

Drogon, Apparently there is an emulation of Mouses available as told to me by Fiona Culloch.

1 Like

Thanks. Yes, @gtoal has mentioned it. I don’t have time or energy to look into it right now though, but it’s nice to know it’s still out there somewhere.

-Gordon