Policy on posts about LLM-assisted retro projects

[EdS]

There are certainly two perspectives, at least, on the subject of LLM-assisted projects, and views can be quite strongly held, whether for or against.

Your moderators have been discussing, for some months, and trying to formulate a policy which is fair and which reflects the values of the forum. It’s been difficult. The result won’t be perfect. But here it is:

The Retro Computing Forum is about the technologies that we love, but more than anything else it’s about the human experience of those technologies, and our relationship to them. With that as our touchstone, we would like to establish some guidelines for large language models (LLMs), generative AI, and services such as chatbots built on these technologies.

• Please contribute authentic experiences and findings, and share them with us.
• Please do not use LLMs or chatbots to compose posts, paste sessions with LLMs, or use LLMs to answer others’ questions.
• Please declare where and how you have used LLMs, chatbots, or coding assistants in any work that you post.
• If you use machine translation or grammar assistance for posting in English, please consider making the comment in your own language, along with a translated or corrected post that you have checked to the best of your ability.

Please be aware that these technologies are controversial and divisive in the wider world – even if you and your circles don’t find them so. This forum is not the place to argue about the pros and cons of these technologies. There are many appropriate places other than this forum to discuss LLMs, generative AI and related technologies, to share creations of or experiences with these things.

As a footnote: I think we’d get the best outcome if each of us could understand and appreciate the other perspective.

Part of that appreciation is to be open about the use of LLMs and to signal that at the level of the topic. We need to make it easy for people who find LLMs distasteful or distressing to avoid threads which are enthusiastic about LLMs.

For that reason we introduce the llm-assisted tag. That’s not a badge of honour, or a badge of shame: it’s a signpost. It also allows us, all of us, to see how often it turns up.

For a civilised discussion from various perspectives, in a retrocomputing context, I refer everyone to the megathread on stardot:
The great AI discussion (PLAY THE BALL, NOT THE PLAYER!)

[HansOtten]

I use LLM’s occasionally to check my writings. Like I use(d) education, university training, dictionaries (books!), spell checkers, grammar checkers, friends and such in the past. And use LLM’s and search engines also to enhance what I am trying to do.with language.

It helps but does not replace my output and I do not feel obliged to have a ‘tag’ on my posts indicating what I use in my retro archeology work.

[EdS]

That sounds fine to me. You’ll continue to have your distinctive voice and to draw on your extensive knowledge!

[mc4004]

I assume Ed is trying to protect our human2human forum from “AI slop.” This is a very important issue. A number of online communities have been seriously compromised, if not destroyed, by this trend. That said, it is sometimes hard to know where to draw the line. Where I’d personally suggest drawing the line is as follows.

We would certainly not welcome postings that are verbatim answers to chatbot prompts. If you didn’t even proofread the generated LLM prose, that’s like a “re-tweet” that you didn’t read fully before forwarding (bad, bad, bad).

To stay within an allowable “gray zone” you need to have written a solid draft of the post yourself, and you can only ask the LLM to “clean it up” in such a way that your voice is still discernible, meaning the structure is still yours, and well over 90% of the words are yours too. No complete rewrites! If you took LLM generated research and heavily edited it, that might be okay (I defer to Ed on this one), but then please use Ed’s new llm-assisted tag. After that you are on the other side of the line. In academic communities, you can’t publishing anything that you didn’t write yourself without attribution anyway. I’d suggest we stay within this spirit of integrity.

I agree with Ed. If English is not your first language, post both your original writing and the English translation. This has the added benefit of connecting you with fellow countrymen who appreciates postings in their native language. Most of my retro collaborators live outside the U.S.

[mc4004]

Maybe we need a machine-translated tag in addition to the llm-assisted tag.

[NoLand]

I think, there’s also another side to this: quite famously, an important aspect of retro computing (esp. 8-bit platforms) is that we’re still able to know our way around these machines and to even determine their state at any time. I’d say, attacking this by LLM-assisted technologies (coding agents and AI harnesses), thus skipping this part entirely, is seriously endangering this and even may spoil it for everyone. Now, the very aspect of achievement (and proof of understanding) is gone, not just for those engaging in these AI-assisted (or even AI-delegated) projects, but also for those engaging with these platforms in a conventional manner. (In other words, the lasting social and cultural cost may by far exceed the momentary benefits.)

[mc4004]

I am sympathetic to your position on process. I understand where you are coming from. At the same time, LLMs offer the possibility of us learning details about retro systems that we didn’t have the manpower to study before. Some reverse-engineering projects need a whole team, but many of us here mostly work alone. A synergy could be truly valuable. Consider this: a living expert teams up with (supervises) an LLM to analyze a particular retro technology and document it for future generations. Even if you do the detective work solo and build up your understanding of how a retro system works, do you have time (or the will) to document it? For that documentation to have value, an LLM needs someone who deeply understands the technology to proofread and correct it.

[EdS]

(Just to note, I’m not expecting to re-litigate this policy, unless perhaps there are some serious shortcomings or major developments to take into account.)

(As the point of the policy is to allow for peaceful co-existence of people who might fall on either side of the great divide, I don’t think it’s helpful to use this thread to make strong position statements of either type. See the stardot thread for such a discussion, and by all means take part in that thread if you wish.)

Edit: note, too, please, that this is a Retrocomputing Forum - don’t make it look like an LLM-usage-discussion forum. Post a retrocomputing topic!

[NoLand]

I can see the use case of coming up with documentation, or bringing old document formats or APIs back to life. It’s always hard to draw a strict line.

Also, much of this isn’t entirely new. There is something, I’d call “augmented retro computing”, around for years, using the leverage of modern technology, where the target system often isn’t much more than merely a terminal. But there’s also yet another blurry line, since maxing out a vintage computer to amounts that would have been hardly affordable or realistic in its heyday is an established part of the hobby.

Personally, what interests me most, apart from the historic aspects, is the challenge of doing things in the respective constraints and limitation of these systems. E.g., doing something that may be perceived as “smooth” with PETSCII characters only. (It’s not that dissimilar from when the Web was still new and doing “smooth things” on a website was an ongoing challenge, a challenge that also advanced the technology. My own way of thinking is not so much, “what if the PET 2001 had sprites, let’s do this!” There had been actually 3rd-party sprite cards using the TMS9918, or cards for redefinable character graphics, or even graphic cards with high-res layers, back then, but they were rare, programs written for these had practically no audience, and I’m frankly more interested in what was available as a stock machine, the mashine that people actually knew.) I also like how systems that may have been frowned upon, back then (mostly because they came late for some reason or another), have collected a decent following, because of these constraints, like the Mattel Aquarius. They are simple, they are fun, sometimes even pretty, and they pose a challenge. What’s not to like?

[p1l0t]

Full disclosure, I was using CONSULT ELIZA as my personal therapist on my Apple IIc back in the 80s before LLMs were even cool. Actually it might be worth some API tokens to introduce her to a modern LLM..