Message boards : Number crunching : Server is out of disk space
| Author | Message |
|---|---|
|
Bedrich Hajek Send message Joined: 28 Mar 09 Posts: 486 Credit: 11,550,461,298 RAC: 4,768,395 Level  Scientific publications      | |
|
Thu 20 Jun 2024 05:49:06 PM EDT | GPUGRID | [error] Error reported by file upload server: Server is out of disk space | |
ID: 61548 | Rating: 0 | rate:
 /   Reply
Quote
| |
|
mrchips Send message Joined: 9 May 21 Posts: 16 Credit: 1,412,539,259 RAC: 47,170 Level  Scientific publications  | |
|
still out of space | |
| ID: 61550 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Bedrich Hajek Send message Joined: 28 Mar 09 Posts: 486 Credit: 11,550,461,298 RAC: 4,768,395 Level Scientific publications | |
|
It's full again. | |
| ID: 61554 | Rating: 0 | rate:
/ Reply
Quote
| |
|
pututu Send message Joined: 8 Oct 16 Posts: 26 Credit: 4,153,801,869 RAC: 14,670 Level  Scientific publications | |
|
Currently, I'm seeing disk full error message. | |
| ID: 61719 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Bedrich Hajek Send message Joined: 28 Mar 09 Posts: 486 Credit: 11,550,461,298 RAC: 4,768,395 Level Scientific publications | |
|
Confirmed. Server is out of disk space. | |
| ID: 61720 | Rating: 0 | rate:
/ Reply
Quote
| |
|
pututu Send message Joined: 8 Oct 16 Posts: 26 Credit: 4,153,801,869 RAC: 14,670 Level Scientific publications | |
|
Seems like it is intermittent, meaning need to perform network transfer retry occasionally to upload the completed tasks. Best to run with a script. | |
| ID: 61722 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Freewill Send message Joined: 18 Mar 10 Posts: 21 Credit: 35,796,571,419 RAC: 56,463,185 Level Scientific publications   | |
|
Still getting this error as of 26 Aug almost 7:00 AM Madrid time. | |
| ID: 61728 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Richard Haselgrove Send message Joined: 11 Jul 09 Posts: 1629 Credit: 9,672,847,368 RAC: 7,829,637 Level  Scientific publications | |
|
26/08/2024 08:58:40 | GPUGRID | [error] Error reported by file upload server: Maintenance underway: file uploads are temporarily disabled. | |
| ID: 61731 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Steve Volunteer moderator Project administrator Project developer Project tester Volunteer developer Volunteer tester Project scientist Send message Joined: 21 Dec 23 Posts: 46 Credit: 0 RAC: 0 Level  Scientific publications | |
|
Thank you for reporting. It is now back and with more space. | |
| ID: 61734 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Bedrich Hajek Send message Joined: 28 Mar 09 Posts: 486 Credit: 11,550,461,298 RAC: 4,768,395 Level Scientific publications | |
|
Server is out of disc space again. | |
| ID: 61779 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Stacie Send message Joined: 29 Mar 20 Posts: 22 Credit: 829,995,819 RAC: 813,869 Level  Scientific publications | |
|
Is this why finished workunits are failing to upload? They are starting to pile up in my que. | |
| ID: 61780 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Stacie Send message Joined: 29 Mar 20 Posts: 22 Credit: 829,995,819 RAC: 813,869 Level Scientific publications | |
|
...do we lose bonus turnaround credit because the upload server is locked up? | |
| ID: 61781 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Erich56 Send message Joined: 1 Jan 15 Posts: 1146 Credit: 11,516,061,501 RAC: 19,931,062 Level Scientific publications | |
...do we lose bonus turnaround credit because the upload server is locked up? yes, at least in the past this has happened. I doubt this has changed now. | |
| ID: 61782 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Erich56 Send message Joined: 1 Jan 15 Posts: 1146 Credit: 11,516,061,501 RAC: 19,931,062 Level Scientific publications | |
|
On August 26 Steve wrote: Thank you for reporting. It is now back and with more space. Steve, now again no finished tasks can be uploaded since last night :-( How come that this happens that often? Is there not any kind of automated reporting early enough (say, once the disk is 80% full), or other measures that prevent this problem to happen every few weeks ? This shouldn't be that difficult to implement. | |
| ID: 61784 | Rating: 0 | rate:
/ Reply
Quote
| |
|
WPrion Send message Joined: 30 Apr 13 Posts: 99 Credit: 3,405,464,157 RAC: 5,326,390 Level Scientific publications | |
Kinda ironic. They are super anxious to get this project completed so they award 10X the points the tasks deserve, therefore attract an army of crunchers, but then don't keep their hardware up to speed to support the effort. | |
| ID: 61785 | Rating: 0 | rate:
/ Reply
Quote
| |
 Steve DoddSend message Joined: 26 Dec 08 Posts: 18 Credit: 4,302,371,348 RAC: 1,527,661 Level Scientific publications | |
|
Oopsie. Just got this message: | |
| ID: 62014 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Bedrich Hajek Send message Joined: 28 Mar 09 Posts: 486 Credit: 11,550,461,298 RAC: 4,768,395 Level Scientific publications | |
|
I got the same message: | |
| ID: 62015 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Erich56 Send message Joined: 1 Jan 15 Posts: 1146 Credit: 11,516,061,501 RAC: 19,931,062 Level Scientific publications | |
|
totally ununderstandable how this can happen so often. | |
| ID: 62016 | Rating: 0 | rate:
/ Reply
Quote
| |
dskagcommunity Send message Joined: 28 Apr 11 Posts: 460 Credit: 889,073,404 RAC: 1,100,130 Level Scientific publications | |
|
After a long time im crunching a bit again and no Uploads since yesterday possible -.- Still the same problems like years before O.o | |
| ID: 62017 | Rating: 0 | rate:
/ Reply
Quote
| |
|
TheFiend Send message Joined: 26 Aug 11 Posts: 100 Credit: 2,604,575,367 RAC: 1,062,119 Level  Scientific publications | |
After a long time im crunching a bit again and no Uploads since yesterday possible -.- Still the same problems like years before O.o I'm another one that has just restarted crunching GPUGRID again after upgrading GPU's.... It's not the only project that encounters this problem... | |
| ID: 62018 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Hi, | |
| ID: 62019 | Rating: 0 | rate:
/ Reply
Quote
| |
|
dskagcommunity Send message Joined: 28 Apr 11 Posts: 460 Credit: 889,073,404 RAC: 1,100,130 Level Scientific publications | |
|
On other projects this is very rarely i must say and im crunching 24/7 since 2011 ^^ But i dont complain too much, just happy this project has WUs again for just ParttimecrunchingPCs. | |
| ID: 62020 | Rating: 0 | rate:
/ Reply
Quote
| |
|
mikey Send message Joined: 2 Jan 09 Posts: 298 Credit: 6,785,855,288 RAC: 3,400,546 Level Scientific publications | |
On other projects this is very rarely i must say and im crunching 24/7 since 2011 ^^ But i dont complain too much, just happy this project has WUs again for just ParttimecrunchingPCs. The problem probably is the fact that the credits are sooo high and the tasks are sooo available that more and more people who wants TONS of credits per day are coming here for them. It also comes down to that as opposed to 6 months ago when several more projects were around to spread the load gpugrid is now one of the few gpu projects around so like the rest is being overwhelmed. I would guess that in time they will figure out how to move things thru the system more quickly and get bigger drivers too and things will be alot better. | |
| ID: 62021 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Greg _BE Send message Joined: 30 Jun 14 Posts: 137 Credit: 122,677,395 RAC: 21,115 Level  Scientific publications | |
|
Probably won't see anything done about it (disk space) until Monday if we are lucky. | |
| ID: 62023 | Rating: 0 | rate:
/ Reply
Quote
| |
|
KeithBriggs Send message Joined: 29 Aug 24 Posts: 37 Credit: 1,645,415,047 RAC: 15,776,688 Level  Scientific publications | |
Probably won't see anything done about it (disk space) until Monday if we are lucky. +1 I started getting errors while computing (EWC) as soon as I started running a script to get them uploaded. There was already a lot of EWC which might contribute to the server overload in the first place. It seems odd that a script would be able to impact a task during what I call the initial cpu-only stage. | |
| ID: 62027 | Rating: 0 | rate:
/ Reply
Quote
| |
|
KeithBriggs Send message Joined: 29 Aug 24 Posts: 37 Credit: 1,645,415,047 RAC: 15,776,688 Level Scientific publications | |
|
Richard Haselgrove identified the issue on the other thread: Corrupt compression likely due to server space but it's also a compression issue. | |
| ID: 62028 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Richard Haselgrove Send message Joined: 11 Jul 09 Posts: 1629 Credit: 9,672,847,368 RAC: 7,829,637 Level Scientific publications | |
|
Some files have started to trickle up, but others are reaching 100% and then sticking again. But it's a sign they may be trying to move some of the results to alternative storage. It'll take time, though. | |
| ID: 62029 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Robertobit Send message Joined: 29 Mar 20 Posts: 3 Credit: 695,184,248 RAC: 390,818 Level  Scientific publications | |
|
Hi I have the same issue | [error] Error reported by file upload server: Server is out of disk space | |
| ID: 62030 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Robertobit Send message Joined: 29 Mar 20 Posts: 3 Credit: 695,184,248 RAC: 390,818 Level Scientific publications | |
|
|me too [error] Error reported by file upload server: Server is out of disk space | |
| ID: 62031 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Khali Send message Joined: 13 Jan 14 Posts: 9 Credit: 388,632,885 RAC: 486,788 Level  Scientific publications | |
|
Just adding in my two cents here. I have files stuck in project back off as well. | |
| ID: 62032 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Erich56 Send message Joined: 1 Jan 15 Posts: 1146 Credit: 11,516,061,501 RAC: 19,931,062 Level Scientific publications | |
Funny how this always seems to happen on a weekend and nothing can be done until Monday. I am wondering anyway how much or not they are interested in keeping their own project working properly, given that obviously everyone leaves on Friday afternoon and no one keeps an eye on what's happening until the following week :-( | |
| ID: 62034 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Greg _BE Send message Joined: 30 Jun 14 Posts: 137 Credit: 122,677,395 RAC: 21,115 Level Scientific publications | |
Funny how this always seems to happen on a weekend and nothing can be done until Monday. EU project (Barcelona). M-F only. Not paid for weekends. They are a donation based program with a bit of government funding, so there is no money for weekend work. So if something goes SOL after hours then it waits until the next working day. Actually I don't think any BOINC project has a 24/7 monitor. Rosetta in the early days did, but all the projects I am on now don't. | |
| ID: 62036 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Erich56 Send message Joined: 1 Jan 15 Posts: 1146 Credit: 11,516,061,501 RAC: 19,931,062 Level Scientific publications | |
Funny how this always seems to happen on a weekend and nothing can be done until Monday. :-( :-( :-( | |
| ID: 62037 | Rating: 0 | rate:
/ Reply
Quote
| |
|
zioriga Send message Joined: 30 Oct 08 Posts: 47 Credit: 567,016,028 RAC: 92,344 Level Scientific publications | |
|
And credits are less rewarded, 'cause of this delay | |
| ID: 62038 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Richard Haselgrove Send message Joined: 11 Jul 09 Posts: 1629 Credit: 9,672,847,368 RAC: 7,829,637 Level Scientific publications | |
|
Phew. All mine have uploaded, reported, and had replacement ATMML tasks issued. | |
| ID: 62045 | Rating: 0 | rate:
/ Reply
Quote
| |
|
roundup  Send message Joined: 11 May 10 Posts: 65 Credit: 10,330,278,875 RAC: 4,674,736 Level Scientific publications | |
|
And again:
| |
| ID: 62070 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Vince Send message Joined: 27 Aug 22 Posts: 1 Credit: 667,195,495 RAC: 453,029 Level Scientific publications | |
|
Server space full no uploads possible | |
| ID: 62071 | Rating: 0 | rate:
/ Reply
Quote
| |
|
roundup Send message Joined: 11 May 10 Posts: 65 Credit: 10,330,278,875 RAC: 4,674,736 Level Scientific publications | |
The focus is on the science behind our projects, not on WU's regular supply. Since 2023, the supply has been much more regular than in many previous years. That has really improved. Also, most of us understand that there are no unlimited resources for most european research institutes to buy or scale hardware. Basically, it's always a good idea to have a backup project that steps in with resource share = 0 whenever there's no work for the main project. | |
| ID: 62072 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Richard Haselgrove Send message Joined: 11 Jul 09 Posts: 1629 Credit: 9,672,847,368 RAC: 7,829,637 Level Scientific publications | |
|
Yes, but ... | |
| ID: 62073 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Freewill Send message Joined: 18 Mar 10 Posts: 21 Credit: 35,796,571,419 RAC: 56,463,185 Level Scientific publications | |
|
On the project's Discord server, one of the admins (giadefa) has posted that "by the end of January we will move to a new server with more disk space." So, that change should address this issue. For now, they seem to be watching as the immediate issue has been resolve. I have been able to upload my completed tasks recently. | |
| ID: 62074 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Michael H.W. Weber Send message Joined: 9 Feb 16 Posts: 73 Credit: 656,229,684 RAC: 130,815 Level Scientific publications | |
|
...the usual problem not solved in years. | |
| ID: 62075 | Rating: 0 | rate:
/ Reply
Quote
| |
Message boards : Number crunching : Server is out of disk space
