Advanced search

Message boards : Wish list : Make Quantum Chemistry working on WSL

Author Message
makracz
Send message
Joined: 9 May 24
Posts: 3
Credit: 1,145,005,000
RAC: 38,777,038
Level
Met
Scientific publications
wat
Message 61916 - Posted: 27 Oct 2024 | 16:35:29 UTC
Last modified: 27 Oct 2024 | 16:47:38 UTC

Over the weekend I've been playing around with Windows WSL (Windows Subsystem for Linux) using Ubuntu 24.04. I managed to get BOINC running and detect NVIDIA GPU. Einstein's GPU tasks (BRP7) ran without any problems and were validated successfully. Unfortunately, Quantum Chemistry tasks fail to compute under those conditions. They threw an error in less than a minute.
My questions are:
- is there a way to run the current QC application on WSL?
- have the developers ever tried to run QC in this environment? Do you think it would be possible to update the QC app to make it run on WSL?

From a user’s perspective setting up the WSL and BOINC installation isn't very challenging. As the QC isn't portable to Windows, this could potentially allow for more hosts to crunch QC tasks.

Erich56
Send message
Joined: 1 Jan 15
Posts: 1131
Credit: 9,960,232,676
RAC: 31,919,313
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwat
Message 61917 - Posted: 27 Oct 2024 | 17:12:13 UTC

my suggestion is: instead of going the WLS detour, QC should be made available for Windows.

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 1335
Credit: 7,538,317,459
RAC: 13,749,196
Level
Tyr
Scientific publications
watwatwatwatwat
Message 61918 - Posted: 27 Oct 2024 | 18:34:49 UTC - in response to Message 61917.
Last modified: 27 Oct 2024 | 18:42:43 UTC

Been explained elsewhere here multiple times. The science app uses external libraries that DON'T HAVE ANY Windows compiled libraries, only Linux ones.

https://www.gpugrid.net/forum_thread.php?id=5429&nowrap=true#61245

So unless the application is completely rewritten to use some other sources that have Windows libraries, this app is never going to be available for Windows hosts.

Ian&Steve C.
Avatar
Send message
Joined: 21 Feb 20
Posts: 1067
Credit: 40,231,533,983
RAC: 1,418
Level
Trp
Scientific publications
wat
Message 61919 - Posted: 27 Oct 2024 | 22:36:23 UTC - in response to Message 61917.

my suggestion is: instead of going the WLS detour, QC should be made available for Windows.


its kind of out of the control of the GPUGrid devs. one piece of required software for the app is not available to Windows. the devs do not maintain this 3rd party software so they cannot just make it work.

they did release a code bounty for anyone capable to port it to Windows. they WANT to have the app available for Windows, but they do not have the ability.

https://github.com/gpugrid/gpugrid/issues/1

the fact that QC doesnt work under WSL is likely a limitation of the Windows nvidia driver. there are some features that only work on the linux driver, and WSL actually uses the Windows driver shared from the Windows environment and not the full Linux driver.
____________

makracz
Send message
Joined: 9 May 24
Posts: 3
Credit: 1,145,005,000
RAC: 38,777,038
Level
Met
Scientific publications
wat
Message 61920 - Posted: 27 Oct 2024 | 22:49:15 UTC
Last modified: 27 Oct 2024 | 23:01:43 UTC

Thanks Keith for pointing to the relevant topic. When I did search for "WSL" I wasn't aware that a regular search on this forum is limited to posts that are no older than 30 days. Should have clicked on the advanced one, my bad.

It seems that although in theory a Linux Distro on WSL should run any Linux native library, there are some problems in this particular case.

the fact that QC doesnt work under WSL is likely a limitation of the Windows nvidia driver. there are some features that only work on the linux driver, and WSL actually uses the Windows driver shared from the Windows environment and not the full Linux driver.

Let's hope that something will change in the future on the WSL/NVIDIA driver end.

[BAT] Svennemans
Send message
Joined: 27 May 21
Posts: 51
Credit: 803,692,017
RAC: 4,229,947
Level
Glu
Scientific publications
wat
Message 61921 - Posted: 28 Oct 2024 | 12:16:24 UTC - in response to Message 61920.
Last modified: 28 Oct 2024 | 12:18:34 UTC

Thanks Keith for pointing to the relevant topic. When I did search for "WSL" I wasn't aware that a regular search on this forum is limited to posts that are no older than 30 days. Should have clicked on the advanced one, my bad.

It seems that although in theory a Linux Distro on WSL should run any Linux native library, there are some problems in this particular case.

the fact that QC doesnt work under WSL is likely a limitation of the Windows nvidia driver. there are some features that only work on the linux driver, and WSL actually uses the Windows driver shared from the Windows environment and not the full Linux driver.

Let's hope that something will change in the future on the WSL/NVIDIA driver end.


WSL runs native Linux apps 'in theory', yes, and for the most part you would be correct. For example, the ATM Linux app runs flawlessly on WSL.
However, WSL is translating native linux systemcalls into Windows systemcalls - unlike a VM which runs as if it were a baremetal linux install - and therein lies the difference between 'in theory' and 'in practice'.
The more deeply an app leverages very specific features of Linux in order to squeeze the last drop of performance from the OS, the more you risk running into cases where a linux syscall doesn't have a 100% windows-supported implementation.
Speaking from experience, in trying to get the QC 3rd-party libraries compiled for windows, I can assure you that these math libraries scrape the absolute bottom of the barrel of what linux can do. As well as leveraging just about every trick in the C99 standard that is not supported by Visual C++.
So that is why they are not supported by WSL, and that is why they're not easily crosscompiled for windows.

Else I would have cashed in that bounty long ago - believe my I tried ;-)

Ian&Steve C.
Avatar
Send message
Joined: 21 Feb 20
Posts: 1067
Credit: 40,231,533,983
RAC: 1,418
Level
Trp
Scientific publications
wat
Message 61922 - Posted: 28 Oct 2024 | 13:32:07 UTC - in response to Message 61921.

Thanks Keith for pointing to the relevant topic. When I did search for "WSL" I wasn't aware that a regular search on this forum is limited to posts that are no older than 30 days. Should have clicked on the advanced one, my bad.

It seems that although in theory a Linux Distro on WSL should run any Linux native library, there are some problems in this particular case.

the fact that QC doesnt work under WSL is likely a limitation of the Windows nvidia driver. there are some features that only work on the linux driver, and WSL actually uses the Windows driver shared from the Windows environment and not the full Linux driver.

Let's hope that something will change in the future on the WSL/NVIDIA driver end.


WSL runs native Linux apps 'in theory', yes, and for the most part you would be correct. For example, the ATM Linux app runs flawlessly on WSL.
However, WSL is translating native linux systemcalls into Windows systemcalls - unlike a VM which runs as if it were a baremetal linux install - and therein lies the difference between 'in theory' and 'in practice'.
The more deeply an app leverages very specific features of Linux in order to squeeze the last drop of performance from the OS, the more you risk running into cases where a linux syscall doesn't have a 100% windows-supported implementation.
Speaking from experience, in trying to get the QC 3rd-party libraries compiled for windows, I can assure you that these math libraries scrape the absolute bottom of the barrel of what linux can do. As well as leveraging just about every trick in the C99 standard that is not supported by Visual C++.
So that is why they are not supported by WSL, and that is why they're not easily crosscompiled for windows.

Else I would have cashed in that bounty long ago - believe my I tried ;-)


great reply, thanks for the added context/info. I was wondering if you had ever tried to port things!

____________

Post to thread

Message boards : Wish list : Make Quantum Chemistry working on WSL

//