Fixing istty() to accept pseudo-terminals too on Uniflex

I noticed when I wrote my little window system and telnetd for the Tektronix 4404 (that both use pseudo-terminals for input) that when I executed “ls” I always got a single column of output - ie ls detects it was running in a script not a terminal.

Decompiling ls and specifically the call early on to istty() shows this:

int isatty(int fd)
  int i;
  stat info;
  i = fstat(fd,&info);
  if (((i == 0) && ((info.st_mode & 0x4f) == 5)) && ((info.m_size & 0xff00) == 0))
    i = 1;
  else {
    i = 0;
  return i;

ie it checks the file descriptor mode is a character file (5) and its size is <256 bytes…

So a pseudo-terminal (mode = 7) and size == 0x100 fails… Doh…

Solution was, I patched 2 bytes in the ls executable to change it to:

if (((i == 0) && ((info.st_mode & 0x4d) == 5)) && ((info.m_size & 0xfe00) == 0))

So it accepts both character terminals (5) AND pseudo terminals (7) as the mask of 0x4d makes then both == 5. Ditto with the m_size masking.

Now ls works as expected when invoked from a pseudo-terminal.


Nice hack!

Note that this kind of buggy behavior and unanticipated consequence (almost specifically!) led Rob Pike to give his talk UNIX Style, or cat -v Considered Harmful, which led to an interesting paper (on linked page). This opinionated position is also the reason that ls on Plan 9 is exclusively single column output, and if you want multi-column output you must use lc, which is a simple rc script (the Plan 9 shell) that dumps the output of ls through mc, whose only job is to columnize its input.

1 Like