Author Topic: Brainstorm: OS/2 on a Different Kernel  (Read 1522 times)

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5228
  • Karma: +44/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Brainstorm: OS/2 on a Different Kernel
« on: March 25, 2025, 11:32:20 pm »
Hello

I was wondering what would be required to run OS/2 over a different kernel.

Let’s theorize, what if we want to have PC that boots Linux, gets to the command line and run something like “startwarp” and the full OS/2 desktop will boot. (Command line, PM, SOM, WPS) and you can run your OS/2 applications there. (Not the 100% of OS/2 applications, but at least start with some)

What would be required to make the OS/2 binaries believe they are running over the OS/2 kernel to run interpretated and not emulating a x86 machine.

From my limited knowledge point of view will be required:

1.- OS2KRNL: The OS/2 kernel has everything that you need on an OS kernel to talk to the hardware, but those will not be necessary since the Linux kernel already does this with the hardware. What would be needed is a layer with all the functions that OS2KRNL provides to DOSCALL1.DLL …. I guess.
As far was a I know OS2KRNL provides a lot of DosXXXX functions to DOSCALL1.DLL. I’m guessing all of this needs to be cloned over Linux (maybe with libc/gcc) to make believe DOSCALL1.DLL that is talking to the kernel.

But I think I’m only talking about the API here, what about the memory management? Where it should go?

Does it make sense?

2.- MOUSE$, KBD$, SCREEN$, CLOCK$. As far as I know this are called “Character Device Drivers”. These are drivers that DOSCALL1.DLL and CPI uses. Here I don’t have the details on how these talks to the CPI. Some layer (DRIVER$ to Linux) to mimic the responses of these driver will be required.

There are a lot more for specific driver like USB, LTP, etc. are also required for software that communicates to hardware, but it had to start with the basic ones to first have a full OS/2 desktop and later improve it with better compatibility.

What would be the vital Character Device Drivers to start with?

Where can I found more information on how CLOCK$ (or any other interacts) with CPI or with any other software that requires it?

3.- Binary compatibility. I don’t know if its completely required that Linux has the “Linear Executable” layer like it was done on the 2ine project. But I think I’m missing this part. I guess that the OS2KRNL provides the instruction to let know that an ,exe file is a “Linear Executable” file, right? Where should that go if we are running a OS2KRNL faked in Linux?

Yes, I know that this theorical version of OS/2 will not be 100% compatible with all software, specially with the software that interacts with hardware. All device driver for OS/2 will be useless, and some new technique will need to be created to grab the Linux drivers and show it on the OS/2 environment (Ex: Linux Audio to UNIAUD1$ bridge). 16 bits may be broken and maybe not required today, VDM will be broken and there will be not full API compatibility first. A proof concept will have an awful compatibility first but it has the potential to grow if it’s open source and if people believe in this kind of long-term strategy.

This will not be easy, because it requires developers with OS/2 and Linux development knowledge to obtain at least a minimum of compatibility to make it usable.

Beside these three components that I theorize, what other thing will be missing to run an OS/2 desktop interpreted in Linux? What do you think? Please correct me where my interpretations of the OS/2 architecture are wrong.

Regards
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5310
  • Karma: +124/-1
    • View Profile
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #1 on: March 26, 2025, 04:26:35 am »
Hi Martin, no expert but my take,
#1, a DosCall1.dll is a big part of the communicating with the kernel, which presents as DosCalls.dll or such. DosCall1.dll would have to communicate with whatever part of Linux manages memory amongst other things.
Also the kernel takes care of loading exe's and dll's, need something to do the same, including doing the fixups. Likely there is WINE code that could be repurposed as Win32 and OS/2 exe's and dll's aren't that different.
#2, guess need to hook into the Linux drivers and present the same stuff in the DLL's as is usual.
#3, I think should be able to load LX binaries, also LE and NE. Once again WINE does similar already, even supports NE binaries I believe.
As for 16 bit, there's still 16bit code in OS/2, especially the low level stuff including DosCall1.dll. WINE runs 16 bit Windows code, perhaps once again ideas from that?
While about it, can look at how WINE supports WIN terminal apps and maybe the same can be done for OS/2 VIO apps. We'd lose full screen support, not much uses it anyways. Linux has a VDM, though not as good as ours as well, or at least it did.
The Presentation Manager would mean mapping PM windows to whatever Linux is using, X11 or Wayland. WPS might be the hardest to support.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5310
  • Karma: +124/-1
    • View Profile
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #2 on: March 26, 2025, 04:42:32 am »
Windows NT, up to W2K or even XP, could run 16 bit OS/2 binaries, text mode out of the box, I ran some. Also they had a Presentation Manager kit to run 16 bit PM applications.
It was interesting how they did it and perhaps some lessons.
Textmode OS/2 1.x ran in a Windows terminal. Other then not respecting the full screen bit, so always in a terminal window, they thought they were running on OS/2. You could do things like use tedit to open C:\config.sys (which only existed virtually) and do things like change the LIBPATH etc and other stuff I assume. Save the file and Windows would update the registry instead of actually writing config.sys.
Never used the PM kit but understand it worked fairly well too, use e.exe to edit the config.sys.
Saw a small article in Byte years ago that MS succeeded in doing the same thing with 32 bit OS/2, including the Presentation Manager. No WPS. As they had the source code and NT did start out as a version of OS/2, they had an advantage compared to us and it still sounded like it was a big job. Guess they were prepared if OS/2 won the OS war.
In a way ReactOS would be a better fit for a kernel then Linux. Otherwise extending WINE, which is not an emulator, might help a lot. The running OS/2 on NT does show it is possible to run our binaries on a different OS and WINE shows running Windows apps on Linux is doable. The WPS is perhaps too hard to run on Linux and maybe even NT.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5228
  • Karma: +44/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #3 on: March 26, 2025, 01:10:19 pm »
The Presentation Manager would mean mapping PM windows to whatever Linux is using, X11 or Wayland. WPS might be the hardest to support.

How does PM connects to the OS/2 kernel currently? Because I would think that PM only goes through CPI to the kernel, and maybe some other DRIVER$, but the idea will be to make PM run through a different kernel and not necessary linking it to other GUI like X11 or Wayland yet. 

Regards
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5310
  • Karma: +124/-1
    • View Profile
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #4 on: March 26, 2025, 04:30:14 pm »
The Presentation Manager would mean mapping PM windows to whatever Linux is using, X11 or Wayland. WPS might be the hardest to support.

How does PM connects to the OS/2 kernel currently? Because I would think that PM only goes through CPI to the kernel, and maybe some other DRIVER$, but the idea will be to make PM run through a different kernel and not necessary linking it to other GUI like X11 or Wayland yet. 

Regards

I don't really know but it has to connect to the video drivers to get access to the frame buffer to create the PM desktop and write/read the framebuffer. Doing things like seamless DOS/WinOS2 support must be tricky too and maybe kernel level.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5228
  • Karma: +44/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #5 on: March 26, 2025, 05:47:36 pm »
The Presentation Manager would mean mapping PM windows to whatever Linux is using, X11 or Wayland. WPS might be the hardest to support.

How does PM connects to the OS/2 kernel currently? Because I would think that PM only goes through CPI to the kernel, and maybe some other DRIVER$, but the idea will be to make PM run through a different kernel and not necessary linking it to other GUI like X11 or Wayland yet. 

Regards

I don't really know but it has to connect to the video drivers to get access to the frame buffer to create the PM desktop and write/read the framebuffer. Doing things like seamless DOS/WinOS2 support must be tricky too and maybe kernel level.

On the very limited/little research I was able to do sometime ago, PMMERGE.DLL uses (or refers to) SINGLEQ$ and KBD$, so maybe SINGLEQ$ is also vital. But I got no reference to something related to video, maybe PMGPI.DLL has some connection to a DRIVER$ that I don't know.

Regards
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5228
  • Karma: +44/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #6 on: April 01, 2025, 04:36:29 pm »
Hello

I'm reading this: Inside the OS/2 Kernel by David C. Zimmerli.
It is from the OS/2 2.11 kernel. It is kind of interesting to see how this author was able to see which files of the kernel source code belongs to which component. (of course he does not has the source code, only the file names). I wonder how he did that, and it makes sense to make the same analysis on the latest ArcaOS kernel.

I still want to try to understand the communication between CPI and OS2KRNL. Just to try to theorize what parts of the OS2KRNL needs to be mimic so DOSCALL1.DLL does not complain (or stop working) because it is on a different kernel.

Regards

Martin Iturbide
OS2World NewsMaster
... just share the dream.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5228
  • Karma: +44/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #7 on: April 01, 2025, 04:44:07 pm »
Hello

I'm trying the NotbookLM AI to explain me some basic things.

Quote
**DOSCALLS1.DLL** is a **dynamic link library (DLL)** in OS/2 that appears to be a **core system component providing a wide range of kernel services and API entry points**.

Here's a breakdown of what the sources indicate about it:

*   **Central System DLL:** Almost all programs utilize **DOSCALLS1.DLL**, suggesting its fundamental role in the OS/2 environment.
*   **Initialization:** It seems to have **instance initialization routines**.
*   **API Entry Points:** **DOSCALL1** contains **16-bit entry points that correspond to various OS/2 APIs**. It also appears to handle **32-bit APIs**, with examples like `Dos32FreeMem`, `DosGetMessage`, and `DosPutMessage` having associated trace events within this DLL.
*   **Exception Handling:** The kernel calls the `Dos32R3ExceptionDispatcher` entry point within **DOSCALLS1.DLL** to **process all user exception handlers**.
*   **Kernel Service Indirection:** Many kernel APIs are **indirected through DOSCALL1.DLL**. This means that calls to certain kernel functions might go through this DLL.
*   **Thunking Layer:** **DOSCALLS1.DLL** includes **UniThunk** trace events, indicating it plays a role in **thunking between 16-bit and 32-bit code**.
*   **Traceability:** The System Tracepoints Reference provides a comprehensive source for deriving API-related breakpoints, often with entry point names prefixed with `pre` or `post`, which is applicable to functions within **DOSCALLS1.DLL**. Numerous trace events are defined for various functions within **DOSCALLS1**, allowing for detailed tracing of API calls.
*   **Relationship with other DLLs:** APIs from other DLLs, such as `QUECALLS.DLL` (queue management APIs), can be **indirected via DOSCALL1**.

In essence, **DOSCALLS1.DLL** serves as a crucial intermediary and provider of fundamental operating system services, handling API calls, managing exceptions, and facilitating communication within the OS/2 environment. Its central role is highlighted by its frequent use across applications and its involvement in various core system functionalities.

Which functions does the OS2KRNL provides that are used by DOSCALLS1.DLL?

Quote
Based on the sources, particularly the trace event information, **OS2KRNL provides a variety of kernel functions that are used by or whose calls are indirected through DOSCALLS1.DLL**. The trace logs in `sg244640.pdf` detail many API calls, specifying a major code for `OS2KRNL` (0x0005) and various minor codes corresponding to specific kernel functions. Many of these APIs have tracepoints that indicate they are indirectly accessed via `DOSCALL1`.

Here are some examples of functions provided by **OS2KRNL** that are relevant to **DOSCALLS1.DLL**, based on the trace events:

*   **Memory Management Functions**:
    *   `Dos32GiveSharedMem`
    *   `Dos32QueryMem`
*   **File System Related Functions**:
    *   `DosBufReset`
    *   `DosCallNmPipe`
    *   `DosDevIoctl`
    *   `DosDisConnectMmPipe`
    *   `DosOpen`
    *   `DosOpen2`
    *   `DosPhysicalDisk`
    *   `DosQNmPipeState`
    *   `DosSetMaxFH`
    *   `DosSetNmPHandState`
    *   `DosSetVec`
*   **Process and Thread Management**:
    *   `DosSystemService`
    *   `DosTimerAsync`
*   **Dynamic Linking and Loading**:
    *   `DosGetModHandle`
    *   `DosGetModName`
    *   `Dos32QueryProcAddr`
*   **Synchronization**:
    *   `DosSetSigHandler`
*   **Handle Management**:
    *   `Dos32IQUERYFHSTATE`
*   **Input/Output**:
    *   `Dos32IRead`
*   **Other**:
    *   `Dos32SetFileLocks`

The "Kernel API Tracepoints Indirected Via DOSCALL1" section further confirms this indirection, although it doesn't list the specific kernel functions themselves but rather the tracepoints associated with the pre-invocation of APIs that are indirected.

In summary, **DOSCALLS1.DLL acts as an intermediary for a significant number of OS/2 kernel functions provided by OS2KRNL**, handling the transition and enabling applications to access these core system services. The trace logs provide concrete examples of this interaction by showing how calls to various kernel functions (residing in **OS2KRNL**) are traced with pre-invocation events often associated with **DOSCALLS1**.
« Last Edit: April 01, 2025, 04:47:26 pm by Martin Iturbide »
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Klafi

  • Newbie
  • *
  • Posts: 6
  • Karma: +1/-0
    • View Profile
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #8 on: April 04, 2025, 10:23:18 pm »
Regarding your question about the communication between CPI (Config.sys Protected-mode Interface) and the OS2KRNL:

CPI provides an interface for the configuration system (config.sys) during boot time and passes information to the kernel before the actual session management (like PM, DOS boxes, etc.) starts. This communication follows a well-defined structure that is processed by the loader—typically involving memory areas, flags, and structured parameter blocks.

The interaction is mostly asymmetrical: CPI delivers startup parameters to the kernel (e.g., via the "init-table" or system objects), but the kernel does not actively call CPI in return. Some modules (such as drivers or IFS components) make use of the provided information via DevHelp or other internal OS/2 APIs, which may be indirectly influenced by CPI.


Relationship between Loader, CPI, and OS2KRNL:
The Loader (OS2LDR) is the first component activated during the boot process.
It loads the kernel (OS2KRNL) as well as other protected-mode modules such as CPI, BASEDEV, IFS, etc.

CPI (Config.sys Protected-mode Interface) is loaded and initialized by the loader.
CPI processes the CONFIG.SYS entries and creates structured information that is essential during system startup.

These pieces of information are passed from CPI to the kernel in the form of:

SysInitData structures Memory blocks / flags / initialization tables
Environment variables and startup parameters (e.g. PROTSHELL, RUN, DEVICE, etc.)

The kernel (OS2KRNL) uses this data to:
Continue system initialization
Properly load drivers and services
Launch the graphical or text-based user environment (e.g., Presentation Manager, DOS sessions)
The communication is one-way – CPI passes data to the kernel, but not the other way around.


References:

IBM Redbook: OS/2 Version 2.1 – Volume 1: Control Program, Chapter 3 – "System Initialization"
https://www.os2museum.com/wp/redbooks/gg243376.pdf

OS/2 Warp Kernel – Theory and Practice by Gary Urban (summary)
https://www.os2museum.com/wp/?p=1906

OS/2 Museum Blog – deep insights into CPI, boot process, kernel internals
https://www.os2museum.com/wp/

IBM OS/2 Warp 4 Device Driver Kit (DDK), includes SysInitData and DevHlp examples
Archived version: https://web.archive.org/...

Alternative approach (modern design principle):
Instead of keeping CPI permanently available, one could introduce a new modular architecture, where a kind of “boot service” or “initialization manager” takes over certain CPI-related functions — but operates in user space, for example under PROTSHELL.


Advantages of a Boot Service in User Space instead of Classic CPI:
Modularity & Extensibility A user-space boot service can be updated or extended more easily, without modifying the kernel or OS2LDR.
New features (such as network initialization or dynamic device management) could be added as loadable modules.

Runtime Intelligence
While CPI only operates during boot time, an initialization service could monitor, adjust, or log system states at runtime.

Improved Error Handling
Errors in CPI often lead to boot failure.
A user-space service can handle such issues more gracefully, write logs, or provide notifications without compromising system stability.

User Interface Capability
A modern initialization service could offer a graphical configuration interface, similar to tools like systemd-analyze on Linux.
This is especially useful for systems like ArcaOS or OS/2 still used in production.

Bidirectional Communication with the Kernel
Using defined APIs, shared memory, or message channels, the service could communicate with the kernel in a controlled way.
This would allow dynamic changes to system parameters or runtime analysis of kernel responses.

Security through Separation
The boot service does not run in kernel mode, which means that any faults or attacks have a lower risk of destabilizing the entire system compared to kernel-level CPI modifications.



I hope I understood the information correctly. I'm no technical genius.
Thanks everyone and for the additions.



Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5228
  • Karma: +44/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #9 on: April 04, 2025, 11:39:37 pm »
In a way ReactOS would be a better fit for a kernel then Linux. Otherwise extending WINE, which is not an emulator, might help a lot. The running OS/2 on NT does show it is possible to run our binaries on a different OS and WINE shows running Windows apps on Linux is doable. The WPS is perhaps too hard to run on Linux and maybe even NT.

On this case 2ine will be an interesting way to go, since it is like WINE but to interpret OS/2 and not Windows. But the issue with 2ine is that the developer tried to replace all possible functions with a cloned open alternative, which makes it a lot of developer work and this is why the project is incomplete (Like cloning all OS/2). What it may be interesting is to use 2ine to clone only the OS2KRNL and Driver$ only and use the IBM's OS/2 binaries for CMD, PM, SOM, WPS, to avoid cloning all OS/2, and find a way to boot the OS/2 desktop over a different kernel.

"Win-OS/2" runs virtualized / Interpreted on OS/2, right?, not emulated. It will be something like having "Warp-Linux VDM" sessions on Linux.

Regards
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5228
  • Karma: +44/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #10 on: April 04, 2025, 11:52:35 pm »
Thanks for the reply Klafli

I want to read your post with more detail.

About this:
Alternative approach (modern design principle):
Instead of keeping CPI permanently available, one could introduce a new modular architecture, where a kind of “boot service” or “initialization manager” takes over certain CPI-related functions — but operates in user space, for example under PROTSHELL.

“Boot service” or “initialization manager” to take over certain CPI-related functions sounds good.
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5310
  • Karma: +124/-1
    • View Profile
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #11 on: April 05, 2025, 02:44:39 am »
In a way ReactOS would be a better fit for a kernel then Linux. Otherwise extending WINE, which is not an emulator, might help a lot. The running OS/2 on NT does show it is possible to run our binaries on a different OS and WINE shows running Windows apps on Linux is doable. The WPS is perhaps too hard to run on Linux and maybe even NT.

On this case 2ine will be an interesting way to go, since it is like WINE but to interpret OS/2 and not Windows. But the issue with 2ine is that the developer tried to replace all possible functions with a cloned open alternative, which makes it a lot of developer work and this is why the project is incomplete (Like cloning all OS/2). What it may be interesting is to use 2ine to clone only the OS2KRNL and Driver$ only and use the IBM's OS/2 binaries for CMD, PM, SOM, WPS, to avoid cloning all OS/2, and find a way to boot the OS/2 desktop over a different kernel.

2ine could be merged into WINE. The reason I mention WINE is that we would need things like a DLL loader, a way to manage shared/private memory and various other things that Windows also does as they are cousins. Seems adapting the WINE DLL loader to load OS/2 DLL's would be simpler then writing one from scratch. Same with managing memory, seems to me that WINE must do memory management in a way similar to OS/2, which is different then Linux usually does and since much of the work has been done to load Win executables and DLL's, adapting would be easier once again then starting from scratch. File system emulation too, OS/2 and Windows expect a drive letter.

Quote
"Win-OS/2" runs virtualized / Interpreted on OS/2, right?, not emulated. It will be something like having "Warp-Linux VDM" sessions on Linux.

Regards

Yes, on X86, Win-OS2 mostly runs virtualized with parts like the mouse emulated by the OS/2 mouse system. The PowerPC OS/2 did emulate Win-OS2 and DOS.
I don't see why we'd need a Win-OS2. WINE runs most Windows binaries fine, there are also options like DOSBox-X that not only run Windows for Workgroups (3.11) but even Win9x.
Similar with DOS. We're not going to be able to run DOS on ring 2 on Linux and that is one big advantage of DOS on OS/2, can use DOS drivers and operate close to the hardware. Linux has DOSEmu and also DOSBox and such.