Fazer Owners Club - Unofficial
General => Your Suggestions and How you can help => Topic started by: simonm on 28 April 2013, 08:18:57 pm
-
I, and a few other foc'cers I've spoken to, have come across slow performance and timeout of page loads on the FOC forum (I've surprisingly got it in the wee hours more often than not). Is it possible to have a look at the logs and/or load to understand if it is performance challenged ?
I think there were a few timeouts connecting to the database too.
Soz if it's an imposition.
Cheers
Simon
-
Hi, thanks for the report. Last year we were having many troubles and eventually our host moved us onto a different server, which seemed to sort the issues. We use shared hosting and so perhaps the other sharers are in a different timezone and busy over(our)night, or maybe there are loads of auto processes firing off.
Would anyone who gets this please make a note of time times and stick them on here (will make for a boring thread but...) and we'll see if we can build up a pattern.
As it's shared hosting I don't have access to the logs but with enough evidence/information I can go to our hosts and they can sort it :thumbup
-
Got one at 18:14 today. Forgot to check the error as I was already refreshing the page (I kinda got used to pages failing to load)
-
12:31 - 12:33
Connection Problems
Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later.
-
that post took about 5 attempts, each failing with the same response.
-
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, webmaster@foc-u.co.uk and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
-
7:04am this morning:
No data received
Unable to load the web page because the server sent no data.
Here are some suggestions:
Reload this web page later.
Error 324 (net::ERR_EMPTY_RESPONSE): The server closed the connection without sending any data.
-
07:12:
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, webmaster@foc-u.co.uk and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
-
I think this could be a reason for some of the double posts that occur via tapatalk (possibly)
I'm also pretty sure these errors are due to the server being overloaded or not powerful enough.
-
See this too
http://foc-u.co.uk/index.php/topic,7559.0.html (http://foc-u.co.uk/index.php/topic,7559.0.html)
-
Farjo, or other admins, do you know what sort of usage stats you have? Bandwidth mostly
I work a lot with hosting, curious to see if I can help out the forum with some alternative hosting :)
-
Farjo, or other admins, do you know what sort of usage stats you have? Bandwidth mostly
I work a lot with hosting, curious to see if I can help out the forum with some alternative hosting :)
I'm glad you mentioned the bit about the bandwidth being due to helping out with hosting. This is definitely not a bandwith issue (IMO) as the web server is responding but falling down a bit further up the architecture (CPU constrained for the 404's and possibly 324's or possibly memory for the database connection resulting in the database/app/pool falling over and restarting)
tbh I don't do web server debugging but I'm relatively confident in my guesses.
ooooooh. A good bloke to know for hosting then :-) :lol :D
-
Failed to connect to forum. Problems persists contact yada yada.
21.39
It was doing it last night as well right up til about midnight when I crashed. O tapatalk
Been fine during the day.
Threads are having the loading swirl for ages as well. Need to back out and try again. Sometimes takes 2 or 3 attempts
-
17:05
Connection ProblemsSorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later.
-
19:45
Error Code 500
Internal Server Error
In my experience usually this is a web server configuration issue
@Simon - Reason I asked about bandwidth is to determine if I could offer something suitable IF the admins wanted to take me up on my offer. It was a long shot as I doubt they would want to move but it was worth a shot...
-
19:45
Error Code 500
Internal Server Error
In my experience usually this is a web server configuration issue
@Simon - Reason I asked about bandwidth is to determine if I could offer something suitable IF the admins wanted to take me up on my offer. It was a long shot as I doubt they would want to move but it was worth a shot...
@Deadeye You're a nice guy trying to help :D
I suspect the 500 is just before the virtual server falls in a heap and restarts (iirc 500 is just something is foobarred but I can't report specifically what it is rather than specifically misconfiguration although that is definitely an option). The forum runs fine most of the time, so I guess the 500 is just appearing just before or after it goes titsup.com. I wouldn't have thought a hosting company would have mis-configured apache/mysql/whatever, but it wouldn't be the first time I was wrong (I would hope for their sake they know their business better than that).
I don't know which of the UKhost4u packages they have signed up for but it'd be interesting to know. I suspect the back up jobs run in the wee hours and suck quite a bit of CPU and memory away from the hosting.
It's all guess work until one of the admins hits the support people with a stick.
-
You are quite correct regarding the 500 errors - what exactly do you do for a living? You have a fair knowledge of these things ;) But yes, generically, 500 is the server threw an unrecoverable error somewhere along the line. The hosting I deal with currently doesn't use VM's so less of an issue of it falling over
Truth be told, I hadn't looked at the current hosting platform that the forum runs on so you have done your research :P As for backups, I'm reserved on that. No way to know for certain as to whether they are running any at all and even then an incremental backup (which is what I would expect) isn't hugely intensive.
-
I've just had a 500 error trying to look at this thread...!!
-
getting a bit frustrating now took me for ever to post a reply on my headlight thead and I got this and lost my post !
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, webmaster@foc-u.co.uk and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
then it didn't look like it was going to let me post this !
-
Got the same thing at 12:20, a 500 internal server error:
EDIT: This was trying to send a PM.
"
Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, webmaster@foc-u.co.uk and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request."
-
Farjo, or other admins, do you know what sort of usage stats you have? Bandwidth mostly
I work a lot with hosting, curious to see if I can help out the forum with some alternative hosting :)
Monthly bandwidth transfer = 6,170.1, disk space usage = 12,500MB (and growing).
Do you work for a hosting company?
-
Have you tried turning it off and turning it back on again :D
-
Have you tried turning it off and turning it back on again :D
:rollin
-
and again! Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, webmaster@foc-u.co.uk and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
-
18:55 - 19:00 "500 Internal Server Error" and generally loading pages slowly.
-
18:55 - 19:00 "500 Internal Server Error" and generally loading pages slowly.
:rollin
I thought we were telling you rather than you telling us :lol
-
:) I thought we were making a record to tell the hosts, and that was the first error I've had.
ps I plan to email them Sunday night so it's on their desk Monday morning, so keep the reports coming.
-
Monday morning cough ! :pokefun bank holiday! :lol
-
:thumbup
-
Connection Problems Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later.
just now !
-
00:30 - several 500 internet server errors. Pages loading very slowly and timing out.
Edit: Including while trying to post this
Chris
-
Internet server errors. Site unusable between 03:30 and 04:15
Chris
Got it again whist typing this post.
-
I am having alot of problems loading pages on the site,,server error, data not found etc, and it seems to be getting worse,just tried to open new members 3 times, no luck.
-
Internal Server ErrorThe server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, webmaster@foc-u.co.uk and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
-
Internal Server ErrorThe server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, webmaster@foc-u.co.uk and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
-
It does seem to occur a lot more in the wee hours. Backups would seem to jump out as the main suspect. I can't understand why incremental backups of files and databases would create so much load. Maybe they don't have a San and are getting network congestion. It all seems quite unlikely for a hosting company as it's fundamentals.
-
The above message slimwilly posted is the one I keep getting too. As Si says it's quite often during the evening and night although has been at other times just not as frequently.
Chris
-
Got one just now.
On a more recent note, anyone else been seeing glitchy rendering on threads (as image below)?
EDIT: Image post screwed up, attempt 2:
-
ISE + 404 error doc @ 12:32. Sent by tapa2
Why is it workin with fone and not pc
(http://img.tapatalk.com/d/13/05/09/3u6y7ypy.jpg)
-
Just got another one.
-
And another.
-
ISE + 404 error doc @ 12:32. Sent by tapa2
Why is it workin with fone and not pc
([url]http://img.tapatalk.com/d/13/05/09/3u6y7ypy.jpg[/url])
I can answer that ;-). The phone app will almost definitely do retries until it gets a success response from the server. A web browser is much more consistent in it's error code handling and will actually tell you when things are going wrong rather than hide you from it.
It's just the way the tapatalk app has been coded. That is also the reason why the tapatalk app can sometimes duplicate post. It's coded differently.
-
About 4-5 times. A minute in total.
Connection Problems Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later.
-
Farjo, or other admins, do you know what sort of usage stats you have? Bandwidth mostly
I work a lot with hosting, curious to see if I can help out the forum with some alternative hosting :)
Monthly bandwidth transfer = 6,170.1, disk space usage = 12,500MB (and growing).
Do you work for a hosting company?
I have a reseller account with unlimited usage at current. I'll run it past my boss to make sure its ok, but the offer is certainly open :) He is one of the Directors for Coreix Ltd - UK hosting provider whom I also work for occasionally as a contractor.
Sorry I didn't get back sooner
-
ISE + 404 error. 06:20
-
500 Internal Server Error again at 00:40 this evening
-
One at 00:10am, and then 3 in a row at 3:10am trying to send a PM.
It's getting a bit ridiculous. :\
-
06:25 failed to connect to forum, although I'd just posted, and pm'd Simon back seconds earlier. :-\
Sent from my HTC One X using Tapatalk 2
-
I've sent Farjo a PM but I haven't heard anything back yet
-
I've given up reporting on the issues they know they're there and happening frequently. It'll all be in the log files if they look.
:b
-
I've given up reporting on the issues they know they're there and happening frequently. It'll all be in the log files if they look.
:b
It's frustrating, as the website grinds to a bit of a halt in the early hours. This is a biker forum! Why can't it filter? :lol :lol
-
Or even better give way to traffic lol
-
Sorry Dead Eye have been away so have only this afternoon replied.
Please keep the reports coming.
-
ISE + 404 error. 01:30 and 01:49
-
Thanks for all the information which I have passed on to our hosts. They are going to move us to a newer server.
-
Thanks for all the information which I have passed on to our hosts. They are going to move us to a newer server.
Great,
Thanks for all your time and work to keep the forum running at it's best Farjo 8)
Chris
-
Error 324 (net::ERR_EMPTY_RESPONSE): The server closed the connection without sending any data.
-
Having problems with very slow loading times and occasionally "cannot display page" screen tonight. See "problems" thread. Others seem to be experiencing the same. Sorry, admins, if this is giving you still more to do! :\
-
And now suddenly seems to be back to normal... :eek
-
I'm happy to look at any logs to help diagnose if there are any. Also there are some tips here (http://wiki.simplemachines.org/smf/Performance_enhancements (http://wiki.simplemachines.org/smf/Performance_enhancements))
I was wondering if http://wiki.simplemachines.org/smf/Performance_enhancements#Cap_the_length_of_your_threads (http://wiki.simplemachines.org/smf/Performance_enhancements#Cap_the_length_of_your_threads) could be an issue.
-
Based on their advice they would recommend capping the thread lengths to ~1000 posts
As far as I am aware there are only a handful of threads that are actually that long. I can only think of one in the 600 forum which is red98's stuttering thread. At the end of the day, software adjustments for a web forum aren't going to help that much when the traffic isn't that high.
-
Based on their advice they would recommend capping the thread lengths to ~1000 posts
As far as I am aware there are only a handful of threads that are actually that long. I can only think of one in the 600 forum which is red98's stuttering thread. At the end of the day, software adjustments for a web forum aren't going to help that much when the traffic isn't that high.
Red's thread is "only" 935 replies long. Even capping the treads to 1000 means there's still room for another 64 replies; a number many threads don't even get near.
How about archiving threads where the last reply was over a year ago, making them read-only? Might that not lighten the load somewhat?
-
Not really
The load is soo low that this is more than likely a network / hardware problem more than anything else. In fact my bet is on the hard drive - its probably too slow to keep up with the number of requests from the SQL server (or equivalent - not sure what is used).
I partly rent a seedbox for torrents (all legal, I promise) but the hard drive is soooo slow that if you start more than 3 or so torrents then the system load rises to something like 10 - this will mean something to simonm more than anyone else I imagine. Essentially a system load of 1 means that everything is being processed as its being received, higher than 1 means that instructions are having to queue before they are dealt with. On most production servers, anything about 1 is usually bad :P Where I work, anything above 0.2 is considered reasonably heavy load
-
How about starting a collection fund to upgrade the hard drives? I don't know how old the tech being used is...
-
That would never happen - its a corporate company who will basically make you upgrade your package on a monthly basis rather than deal with updating old hardware.
In any case, I spoke to Farjo about potentially moving the site but I think they are happy with where they are
-
Based on their advice they would recommend capping the thread lengths to ~1000 posts
As far as I am aware there are only a handful of threads that are actually that long. I can only think of one in the 600 forum which is red98's stuttering thread. At the end of the day, software adjustments for a web forum aren't going to help that much when the traffic isn't that high.
I find that section a bit ambiguous to my understanding. I would have thought, that post size and content would be very relevant too. If every post had an image and some web links and those pictures were in different directories and the word censor had to work on every youtube url and 30 members were browsing a different thread I could see there being a large load on the disk I/O, especially if the database sits on the same drive. Even with a 10k HDD the read and write head would be flying all over the shop. Even with db and disk caching I can see performance being an issue.
Having the avatars in a single directory could help (if it isn't done already) http://wiki.simplemachines.org/smf/Performance_enhancements#Move_your_uploaded_avatars_directory (http://wiki.simplemachines.org/smf/Performance_enhancements#Move_your_uploaded_avatars_directory)
as would having all the images in a single directory.
Obviously as I'm not an admin I can't tell if all of this is already done. I'm sure the admins have already done it (or it was probably standardised/bugfixed in a SMF release if it had that much of an impact) but without debug and logging it's hard to tell. If the ISP has moved machines to a higher spec machine (which I believe they did) then it could move the performance bottleneck somewhere else and turn it in to a sporadic or environmentally timed issue.
Not that it matters I think my 'help' isn't particularly wanted or needed so it's time to bow out ;)
-
Its more to do with where the data is stored on the hard drive (fragmentation)
Lots of tiny pieces of data (each post for example) requires a different read request if its not stored exactly next to another piece being requested (which is unlikely). Therefore you need an HDD with a very amount of IOPS more than transfer rates. This is where RAID arrays and SSDs come in to increase performance and yield higher IOPS to the OS / Applications
Its worth noting that having all the avatars in a single directory isn't going to make any difference to data fragmentation unless I'm missing something.
-
Its more to do with where the data is stored on the hard drive (fragmentation)
Lots of tiny pieces of data (each post for example) requires a different read request if its not stored exactly next to another piece being requested (which is unlikely). Therefore you need an HDD with a very amount of IOPS more than transfer rates. This is where RAID arrays and SSDs come in to increase performance and yield higher IOPS to the OS / Applications
Its worth noting that having all the avatars in a single directory isn't going to make any difference to data fragmentation unless I'm missing something.
You only get fragmentation when you're deleting/overwriting data, if you're only adding it (which will mainly be the case on a forum) you shouldn't see much apart from possibly on the database files. On a unix box with ext3/4 you'll get virtually no fragmentation anyway. I don't believe fragmentation is much of a problem except in windows fat/fat32/NTFS environments. In conclusion I don't think fragmentation is an issue at all but we're all welcome to our opinions and I could be wrong.
Damn... I'm replying when I said I wouldn't. I could argue at length with you on the pros and cons of RAID0/5/6 etc but it's all environment/hardware specific so it'd all be guessing and when we don't know the set-up it's impossible and pointless to say. If a SMF forum needs a RAID array or SSD then you'd better be running the Ubuntu forums otherwise it's a hell of a spec machine just to run some php and mysql to post up some images.
Maybe you're right, maybe I'm wrong. Dunno.
-
I just figured out you're talking about database fragmentation rather than filesystem fragmentation.
Database fragmentation should really be hidden from the end user due to database/filesystem caching, the efficient use of stored procedures and the fact that very little data is deleted from the database due to the fact that the forum is adding rows rather than deleting them. I'm pretty certain that this won't be an issue on a forum of this size, but I'm happy to be proven incorrect if it fixes the problem.
Databases. You love em or you hate em.
-
I'm not arguing, I find this sort of stuff fascinating believe or not and it's always good to bounce ideas off of people. I know for a fact that if I hadn't had the support of my fellow class mates at uni to throw ideas around, none of us would have done as well as we did in the end :)
Fragmentation can also occur when the disk drive is particularly full. Your are right though, most modern OS's and file systems will sensibly organise data and will periodically attempt to de-fragment what it can. I still think there is an issue with lots of small bits of data being collected from lots of different places on the hard drive though. If you think about it, information relating to the user is probably in one db table, then there is data about the post, the thread, avatars, logos, different scripts and includes, icons and so on. Its a big toll
However, even a crappy little netbook (the tiny 10" things with no optical drive) could probably run this without much issue. The problem is when you start tacking on lots of forums and other websites and resource intensive applications. The end result is that the poor hard drive takes one hell of a beating in terms of read / write requests where as data throughput is probably 10-20% of its maximum capacity.
As I was writing this, your second post has arrived (email alerts ftw). I was originally talking about file systems as fragmentation in the database is usually not significant for a forum - your are correct in that DELETE requests are a complete minority. I would suggest probably 80%+ is SELECT, the rest being UPDATE and INSERT a majority of which is probably UPDATE.
I don't think there is much that can be done to make the site faster since Farjo doesn't control the server and its setup - they rent hosting. It's just an interesting topic to discuss in a hypothetical world :D
-
It's just an interesting topic to discuss in a hypothetical world :D
No it isnt...its foccin borin. Yawn
-
Only to the likes of you ;)
-
It's just an interesting topic to discuss in a hypothetical world :D
No it isnt...its foccin borin. Yawn
The "next topic" link is at the top and bottom of each thread...
-
If you think about it, information relating to the user is probably in one db table, then there is data about the post, the thread, avatars, logos, different scripts and includes, icons and so on. Its a big toll
I don't think there is much that can be done to make the site faster since Farjo doesn't control the server and its setup - they rent hosting. It's just an interesting topic to discuss in a hypothetical world :D
It might be http://www.ukhost4u.co.uk/shared-web-hosting/basic (http://www.ukhost4u.co.uk/shared-web-hosting/basic) (I suspect the data is over a gig, but doubtful over 10 (mainly pictures I'd say). This gives you the ability to install SMF from a script and use mysql. In this case, although the admins wouldn't have control over the server's set up, I suspect they would have control over SMF's config but probably not ssh/root access. The awstats and webalyzer stats look like cool utilities though.
I'd guess the avatars, icons and pictures would be stored on disk as blob's in a database are generally bad news if you can serve them up in a simpler way you'd be advised to. If I'd written it I'd not store significant binary data in a database table if I could possibly help it. Think of the traffic flow http -> disk vs http -> database ->disk
-
Im with Exupnut on this one!!!!!!! :z
-
Im with Exupnut on this one!!!!!!! :z
I.T. guys can be boring, we're a misunderstood lot (me more than most) :). I'd recommend an unnotify click :rolleyes
-
Id recommend we stick to talking bikes!!!!!!
-
Id recommend we stick to talking bikes!!!!!!
Bugger. I know nowt about bikes. Only had one for 6 months.
-
I find it interesting even though I don't know what half of it means lol.
-
If you think about it, information relating to the user is probably in one db table, then there is data about the post, the thread, avatars, logos, different scripts and includes, icons and so on. Its a big toll
I don't think there is much that can be done to make the site faster since Farjo doesn't control the server and its setup - they rent hosting. It's just an interesting topic to discuss in a hypothetical world :D
It might be [url]http://www.ukhost4u.co.uk/shared-web-hosting/basic[/url] ([url]http://www.ukhost4u.co.uk/shared-web-hosting/basic[/url]) (I suspect the data is over a gig, but doubtful over 10 (mainly pictures I'd say). This gives you the ability to install SMF from a script and use mysql. In this case, although the admins wouldn't have control over the server's set up, I suspect they would have control over SMF's config but probably not ssh/root access. The awstats and webalyzer stats look like cool utilities though.
I'd guess the avatars, icons and pictures would be stored on disk as blob's in a database are generally bad news if you can serve them up in a simpler way you'd be advised to. If I'd written it I'd not store significant binary data in a database table if I could possibly help it. Think of the traffic flow http -> disk vs http -> database ->disk
From information that Farjo has given me, I believe they are on a higher package - the site is growing quickly and will soon pass the 10GB disk limit
I would find it unlikely for the avatars to be stored as blobs but could be wrong - seems a very strange approach. In any case, UKHost4U have been described to me as "ok". Their support seems ok based on what the admins have mentioned but I expect they are overloading their servers with users for monetary gain which is the cause of the slowness. Farjo mentioned about a New Server in the General section - I guess they just moved the account which is a fairly straight forward process and hope that the new one has less intensive users. Its hit and miss at the end of the day
-
I did nt want a nother thead just for this but I am still getting issues ?
I have had this afew times now ! Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, webmaster@foc-u.co.uk and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
-
Thanks for reporting it. How long did they last - were you able to refresh and get the site up or was it unavailable for a while?
-
it was simple just refresh once or twice I had to go back to my home page ( google ) and select the link again from my favourites
but nothing to drastic
no clearing out of the cashe or rebooting or having to log in again
-
OK thanks :thumbup
-
A few more issues it seems - site was inaccessible for 10-15 mins with this being the response
-
Thank you :thumbup