Page Index Toggle Pages: 1 ReplyAdd Poll Send Topic
Hot Topic (More than 10 Replies) Backing up large forum ends prematurely (Read 8862 times)
 
Paste Member Name in Quick Reply Box Monni
Language
***
Offline


Min izāmō

Posts: 413
Location: Kaarina, Finland
Joined: Jul 16th, 2014
Gender: Male
Mood: Frustrated
Zodiac sign: Pisces
Re: Backing up large forum ends prematurely
Reply #14 - Aug 4th, 2014 at 4:37pm
Mark & QuoteQuote  
This is becoming more of a feature enhancement request than a plain bug report Smiley Heading for 2.6.1 or 2.6.2 Wink
  
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box Dandello
Forum Administrator
YaBB Modder
*****
Offline


I love YaBB 2.7!

Posts: 2276
Location: The Land of YaBB
Joined: Feb 12th, 2014
Gender: Female
Mood: Annoyed
Zodiac sign: Virgo
Re: Backing up large forum ends prematurely
Reply #12 - Aug 4th, 2014 at 4:30pm
Mark & QuoteQuote  
Keeping the last 2 or 3 backups is probably all you need. And even taking an hour for a BIG backup isn't bad. (YaBBForum used to take several hours on its old host.) So if you can use cPanel for the backups, you should proabably do it that way.  (But pointing out the XMLHttpRequest process being used by cPanel might point YaBB in a better direction for backing up.  Smiley)
  

Perfection is not possible. Excellence, however, is excellent.
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Monni
Language
***
Offline


Min izāmō

Posts: 413
Location: Kaarina, Finland
Joined: Jul 16th, 2014
Gender: Male
Mood: Frustrated
Zodiac sign: Pisces
Re: Backing up large forum ends prematurely
Reply #11 - Aug 4th, 2014 at 4:10pm
Mark & QuoteQuote  
I think it's hitting a limit on CPU usage, because the www server seems very unresponsive during backup... Constantly downloading site backups seems quite a dumb idea because of the large size... It would fill up any normal hard disk in just a few days unless I remove all the old backups except maybe last 3. Last time I tried zipping up the forum using cPanel it took about an hour to finish, but that's because I had to actually download the individual zip files to local machine. Without downloading, I assume it would take only about 30 seconds for all YaBB folders. Only difference between cPanel and YaBB itself I think is that cPanel uses XMLHttpRequest for backing up.

PS. I posted this also on the yabbforum.com community forum Wink
  
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box Dandello
Forum Administrator
YaBB Modder
*****
Offline


I love YaBB 2.7!

Posts: 2276
Location: The Land of YaBB
Joined: Feb 12th, 2014
Gender: Female
Mood: Annoyed
Zodiac sign: Virgo
Re: Backing up large forum ends prematurely
Reply #10 - Aug 4th, 2014 at 3:49pm
Mark & QuoteQuote  
If the other archiving methods didn't work then I strongly suspect the problem is not with YaBB's backup but with how the server is configured to handle archiving. Not having access to the host is going to make it tough as I think this is something that has to be looked at by the hosting service. My best guess is there's some sort of limit you're hitting - memory, CPU or a software setting - that's stopping the zip at 13MB. You may need to ask the site owner to make sure there are site backups being made on the server and that those are being downloaded for safekeeping.

(I know of one shared hosting service - not a cheap one, either - that offers free site backups but they don't tell their clients that those backups are only saved for 12 hours unless you pay them big money to email them to you - or you go online every 12 hours to grab the newest backup before they delete it.  >Sad )
  

Perfection is not possible. Excellence, however, is excellent.
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Monni
Language
***
Offline


Min izāmō

Posts: 413
Location: Kaarina, Finland
Joined: Jul 16th, 2014
Gender: Male
Mood: Frustrated
Zodiac sign: Pisces
Re: Backing up large forum ends prematurely
Reply #9 - Aug 4th, 2014 at 3:28pm
Mark & QuoteQuote  
Dandello wrote on Aug 4th, 2014 at 2:50pm:
Do you have access to tar? Do you have access to any server backup utilities?
It may simply be that zip running through YaBB on that server can't handle that big of a folder if it's timing out at 13MB.

You might also want to post this on YaBBForum so JonB can see it. He's done extensive testing with Backup and may have some ideas on what's going on.


I tried all the available archiving methods, zip was the only one that even remotely worked... It's not my hosting account, so I just can't test every single possible backup method. And I really don't like staying up 24/7 whole week to wait for the forum to be absolutely quiet so I can shut it down for hours.
  
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box Dandello
Forum Administrator
YaBB Modder
*****
Offline


I love YaBB 2.7!

Posts: 2276
Location: The Land of YaBB
Joined: Feb 12th, 2014
Gender: Female
Mood: Annoyed
Zodiac sign: Virgo
Re: Backing up large forum ends prematurely
Reply #8 - Aug 4th, 2014 at 2:50pm
Mark & QuoteQuote  
Do you have access to tar? Do you have access to any server backup utilities?
It may simply be that zip running through YaBB on that server can't handle that big of a folder if it's timing out at 13MB.

You might also want to post this on YaBBForum so JonB can see it. He's done extensive testing with Backup and may have some ideas on what's going on.
  

Perfection is not possible. Excellence, however, is excellent.
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Monni
Language
***
Offline


Min izāmō

Posts: 413
Location: Kaarina, Finland
Joined: Jul 16th, 2014
Gender: Male
Mood: Frustrated
Zodiac sign: Pisces
Re: Backing up large forum ends prematurely
Reply #7 - Aug 4th, 2014 at 2:41pm
Mark & QuoteQuote  
I was doing testing and tweaking... Not to mention previous full backups were bad due to bugs in the actual backup code like I mentioned. It really doesn't matter if I try to zip sources and data separately if it backups only 4-13 MB of 560 MB.
  
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box Dandello
Forum Administrator
YaBB Modder
*****
Offline


I love YaBB 2.7!

Posts: 2276
Location: The Land of YaBB
Joined: Feb 12th, 2014
Gender: Female
Mood: Annoyed
Zodiac sign: Virgo
Re: Backing up large forum ends prematurely
Reply #6 - Aug 4th, 2014 at 1:48pm
Mark & QuoteQuote  
Obviously the problem with zip isn't just the typo in that folder name. Since you (I assume) know the problem is stemming from the ulpd section, maybe zipping that separately will do the trick. (That is one of the suggestions for large forums having problems with backing up - zip things by sections or use tar. You're just showing different symptoms - the server isn't throwing a server error or a blank screen.)

Another bit of advice - the Sources, Languages, Templates and html sections don't change much unless you're doing testing, tweaking, or loading mods. That leaves Boards, Members, Messages, Variables, and Uploads as the ones that are the most critical to restoring a damaged forum.

Run two or more batches - the first one with the 'fixed' data can be run once a month or even less often. The second one is the one that needs to be run to safeguard your forum.

(JonB and I have been fighting with Backup for years - seriously, since before 2.5.2 came out - and we haven't come up with any good fixes. )

Edited:
Yes, I know this isn't what you wanted to hear. But YaBB's Backup utility bears a strong resemblance to a 72 Datsun. If you want to take it cross country make sure you carry lots of water and don't do it a hot day without planning lots of stops.  Wink
  

Perfection is not possible. Excellence, however, is excellent.
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Monni
Language
***
Offline


Min izāmō

Posts: 413
Location: Kaarina, Finland
Joined: Jul 16th, 2014
Gender: Male
Mood: Frustrated
Zodiac sign: Pisces
Re: Backing up large forum ends prematurely
Reply #5 - Aug 4th, 2014 at 9:25am
Mark & QuoteQuote  
Dandello wrote on Aug 4th, 2014 at 5:16am:
The Bookmark error is in line 732 of Admin/Backup.pm. (Yes, it should be Bookmark.)
See if fixing that fixes the main problem.


I already tried that... third attempt resulted in complete zip file... first two attempts stopped around 4 MB. I tried waiting several minutes but the files didn't grow any bigger so it wasn't silently appending more chunks to the zip file anymore. I'm thinking it should actually reload the page between the chunks so it wouldn't kill the server with too high cpu load.
  
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box Dandello
Forum Administrator
YaBB Modder
*****
Offline


I love YaBB 2.7!

Posts: 2276
Location: The Land of YaBB
Joined: Feb 12th, 2014
Gender: Female
Mood: Annoyed
Zodiac sign: Virgo
Re: Backing up large forum ends prematurely
Reply #4 - Aug 4th, 2014 at 5:16am
Mark & QuoteQuote  
The Bookmark error is in line 732 of Admin/Backup.pm. (Yes, it should be Bookmark.)
See if fixing that fixes the main problem.
  

Perfection is not possible. Excellence, however, is excellent.
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Monni
Language
***
Offline


Min izāmō

Posts: 413
Location: Kaarina, Finland
Joined: Jul 16th, 2014
Gender: Male
Mood: Frustrated
Zodiac sign: Pisces
Re: Backing up large forum ends prematurely
Reply #3 - Aug 4th, 2014 at 4:01am
Mark & QuoteQuote  
Dandello wrote on Aug 4th, 2014 at 12:46am:
There's also an ongoing issue with the zip process throwing errors when a folder is empty or nothing has changed in it.


That's the downside of splitting the backup process to chunks... Some things can be fixed, some things can't be... But what really hurts is that in current state it only works every fifth time even when doing full backup and not an incremental one.
  
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box Dandello
Forum Administrator
YaBB Modder
*****
Offline


I love YaBB 2.7!

Posts: 2276
Location: The Land of YaBB
Joined: Feb 12th, 2014
Gender: Female
Mood: Annoyed
Zodiac sign: Virgo
Re: Backing up large forum ends prematurely
Reply #2 - Aug 4th, 2014 at 12:46am
Mark & QuoteQuote  
There's also an ongoing issue with the zip process throwing errors when a folder is empty or nothing has changed in it.
  

Perfection is not possible. Excellence, however, is excellent.
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Dandello
Forum Administrator
YaBB Modder
*****
Offline


I love YaBB 2.7!

Posts: 2276
Location: The Land of YaBB
Joined: Feb 12th, 2014
Gender: Female
Mood: Annoyed
Zodiac sign: Virgo
Re: Backing up large forum ends prematurely
Reply #1 - Aug 4th, 2014 at 12:32am
Mark & QuoteQuote  
That could be. I know the the backup utility has some 'issues' and JonB has worked on it. (But he hasn't had time to give me his suggestions.)
  

Perfection is not possible. Excellence, however, is excellent.
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Monni
Language
***
Offline


Min izāmō

Posts: 413
Location: Kaarina, Finland
Joined: Jul 16th, 2014
Gender: Male
Mood: Frustrated
Zodiac sign: Pisces
Backing up large forum ends prematurely
Aug 3rd, 2014 at 10:49pm
Mark & QuoteQuote  
I have a large forum with lots of attachments...

When I try to backup it using "zip" method, 4 times out of 5, it stops after about 10 MB... Usually it works after I resave the backup settings even if I haven't changed any of the settings. When it finally works, the browser starts downloading the page instead of showing it...

the resulting file reads:

Code
Select All
 zip error: Nothing to do! (try: zip -gqr /home/sg120/public_html/cgi-bin/yabb2/Backups/backup.1407103604.src_bo_lan_mem_m
es_temp_var_html_upld.zip . -i /home/sg120/public_html/yabbfiles/BookMarks)
  



I think the error is because it should be "yabbfiles/Bookmarks" instead (lowercase "m").
  
Back to top
IP Logged
 
Page Index Toggle Pages: 1
ReplyAdd Poll Send Topic
Bookmarks: del.icio.us Digg Facebook Google LinkedIn reddit Twitter Yahoo
Backing up large forum ends prematurely

Please type the characters exactly as they appear in the image,
without the first 2 and last 2 characters.
The characters must be typed in the same order,
and they are case-sensitive.
Open Preview Preview

You can resize the textbox by dragging the right or bottom border.
Off Topic Comment Insert Spoiler
Insert Hyperlink Insert FTP Link Insert Image Insert E-mail Insert Media Insert Table Insert Table Row Insert Table Column Insert Horizontal Rule Insert Teletype Insert Code Insert Quote Edited Superscript Subscript Insert List /me - my name Insert Marquee Insert Timestamp No Parse
Bold Italicized Underline Insert Strikethrough Highlight
                       
Change Text Color
Insert Preformatted Text Left Align Centered Right Align
resize_wb
resize_hb







Max 5000 characters. Remaining characters:
Text size: %
More Smilies
View All Smilies
Collapse additional features Collapse/Expand additional features Smiley Wink Cheesy Grin Angry Sad Shocked Cool Huh Roll Eyes Tongue Embarrassed Lips Sealed Undecided Kiss Cry