Author Topic: Performance (Read 16065 times)

ChristoT · « **Reply #50 on:** 17 May 2013, 02:12:03 pm »

Quote from: simonm on 17 May 2013, 02:03:16 pm

I've given up reporting on the issues they know they're there and happening frequently. It'll all be in the log files if they look.

It's frustrating, as the website grinds to a bit of a halt in the early hours. This is a biker forum! Why can't it filter?

Exupnut · « **Reply #51 on:** 17 May 2013, 02:48:28 pm »

Or even better give way to traffic lol

Farjo · « **Reply #52 on:** 17 May 2013, 03:32:06 pm »

Sorry Dead Eye have been away so have only this afternoon replied.

Please keep the reports coming.

Chris · « **Reply #53 on:** 19 May 2013, 01:40:19 am »

ISE + 404 error. 01:30 and 01:49

Farjo · « **Reply #54 on:** 20 May 2013, 04:35:34 am »

Thanks for all the information which I have passed on to our hosts. They are going to move us to a newer server.

Chris · « **Reply #55 on:** 20 May 2013, 09:35:53 am »

Quote from: Farjo on 20 May 2013, 04:35:34 am

Thanks for all the information which I have passed on to our hosts. They are going to move us to a newer server.

Great,

Thanks for all your time and work to keep the forum running at it's best Farjo

Chris

simonm · « **Reply #56 on:** 02 July 2013, 10:08:28 pm »

Error 324 (net::ERR_EMPTY_RESPONSE): The server closed the connection without sending any data.

Hedgetrimmer · « **Reply #57 on:** 02 July 2013, 10:21:13 pm »

Having problems with very slow loading times and occasionally "cannot display page" screen tonight. See "problems" thread. Others seem to be experiencing the same. Sorry, admins, if this is giving you still more to do! $:\$

Hedgetrimmer · « **Reply #58 on:** 02 July 2013, 11:07:13 pm »

And now suddenly seems to be back to normal...

simonm · « **Reply #59 on:** 03 July 2013, 11:51:33 am »

I'm happy to look at any logs to help diagnose if there are any. Also there are some tips here (http://wiki.simplemachines.org/smf/Performance_enhancements)

I was wondering if http://wiki.simplemachines.org/smf/Performance_enhancements#Cap_the_length_of_your_threads could be an issue.

Dead Eye · « **Reply #60 on:** 03 July 2013, 12:13:11 pm »

Based on their advice they would recommend capping the thread lengths to ~1000 posts

As far as I am aware there are only a handful of threads that are actually that long. I can only think of one in the 600 forum which is red98's stuttering thread. At the end of the day, software adjustments for a web forum aren't going to help that much when the traffic isn't that high.

ChristoT · « **Reply #61 on:** 03 July 2013, 01:48:58 pm »

Quote from: Dead Eye on 03 July 2013, 12:13:11 pm

Based on their advice they would recommend capping the thread lengths to ~1000 posts

As far as I am aware there are only a handful of threads that are actually that long. I can only think of one in the 600 forum which is red98's stuttering thread. At the end of the day, software adjustments for a web forum aren't going to help that much when the traffic isn't that high.

Red's thread is "only" 935 replies long. Even capping the treads to 1000 means there's still room for another 64 replies; a number many threads don't even get near.

How about archiving threads where the last reply was over a year ago, making them read-only? Might that not lighten the load somewhat?

Dead Eye · « **Reply #62 on:** 03 July 2013, 02:03:07 pm »

Not really

The load is soo low that this is more than likely a network / hardware problem more than anything else. In fact my bet is on the hard drive - its probably too slow to keep up with the number of requests from the SQL server (or equivalent - not sure what is used).

I partly rent a seedbox for torrents (all legal, I promise) but the hard drive is soooo slow that if you start more than 3 or so torrents then the system load rises to something like 10 - this will mean something to simonm more than anyone else I imagine. Essentially a system load of 1 means that everything is being processed as its being received, higher than 1 means that instructions are having to queue before they are dealt with. On most production servers, anything about 1 is usually bad :P Where I work, anything above 0.2 is considered reasonably heavy load

ChristoT · « **Reply #63 on:** 03 July 2013, 02:26:42 pm »

How about starting a collection fund to upgrade the hard drives? I don't know how old the tech being used is...

Dead Eye · « **Reply #64 on:** 03 July 2013, 02:28:04 pm »

That would never happen - its a corporate company who will basically make you upgrade your package on a monthly basis rather than deal with updating old hardware.

In any case, I spoke to Farjo about potentially moving the site but I think they are happy with where they are

simonm · « **Reply #65 on:** 03 July 2013, 04:08:21 pm »

Quote from: Dead Eye on 03 July 2013, 12:13:11 pm

Based on their advice they would recommend capping the thread lengths to ~1000 posts

As far as I am aware there are only a handful of threads that are actually that long. I can only think of one in the 600 forum which is red98's stuttering thread. At the end of the day, software adjustments for a web forum aren't going to help that much when the traffic isn't that high.

I find that section a bit ambiguous to my understanding. I would have thought, that post size and content would be very relevant too. If every post had an image and some web links and those pictures were in different directories and the word censor had to work on every youtube url and 30 members were browsing a different thread I could see there being a large load on the disk I/O, especially if the database sits on the same drive. Even with a 10k HDD the read and write head would be flying all over the shop. Even with db and disk caching I can see performance being an issue.

Having the avatars in a single directory could help (if it isn't done already) http://wiki.simplemachines.org/smf/Performance_enhancements#Move_your_uploaded_avatars_directory

as would having all the images in a single directory.

Obviously as I'm not an admin I can't tell if all of this is already done. I'm sure the admins have already done it (or it was probably standardised/bugfixed in a SMF release if it had that much of an impact) but without debug and logging it's hard to tell. If the ISP has moved machines to a higher spec machine (which I believe they did) then it could move the performance bottleneck somewhere else and turn it in to a sporadic or environmentally timed issue.

Not that it matters I think my 'help' isn't particularly wanted or needed so it's time to bow out

Dead Eye · « **Reply #66 on:** 03 July 2013, 04:12:44 pm »

Its more to do with where the data is stored on the hard drive (fragmentation)

Lots of tiny pieces of data (each post for example) requires a different read request if its not stored exactly next to another piece being requested (which is unlikely). Therefore you need an HDD with a very amount of IOPS more than transfer rates. This is where RAID arrays and SSDs come in to increase performance and yield higher IOPS to the OS / Applications

Its worth noting that having all the avatars in a single directory isn't going to make any difference to data fragmentation unless I'm missing something.

simonm · « **Reply #67 on:** 03 July 2013, 04:24:31 pm »

Quote from: Dead Eye on 03 July 2013, 04:12:44 pm

Its more to do with where the data is stored on the hard drive (fragmentation)

Lots of tiny pieces of data (each post for example) requires a different read request if its not stored exactly next to another piece being requested (which is unlikely). Therefore you need an HDD with a very amount of IOPS more than transfer rates. This is where RAID arrays and SSDs come in to increase performance and yield higher IOPS to the OS / Applications

Its worth noting that having all the avatars in a single directory isn't going to make any difference to data fragmentation unless I'm missing something.

You only get fragmentation when you're deleting/overwriting data, if you're only adding it (which will mainly be the case on a forum) you shouldn't see much apart from possibly on the database files. On a unix box with ext3/4 you'll get virtually no fragmentation anyway. I don't believe fragmentation is much of a problem except in windows fat/fat32/NTFS environments. In conclusion I don't think fragmentation is an issue at all but we're all welcome to our opinions and I could be wrong.

Damn... I'm replying when I said I wouldn't. I could argue at length with you on the pros and cons of RAID0/5/6 etc but it's all environment/hardware specific so it'd all be guessing and when we don't know the set-up it's impossible and pointless to say. If a SMF forum needs a RAID array or SSD then you'd better be running the Ubuntu forums otherwise it's a hell of a spec machine just to run some php and mysql to post up some images.

Maybe you're right, maybe I'm wrong. Dunno.

simonm · « **Reply #68 on:** 03 July 2013, 04:31:50 pm »

I just figured out you're talking about database fragmentation rather than filesystem fragmentation.

Database fragmentation should really be hidden from the end user due to database/filesystem caching, the efficient use of stored procedures and the fact that very little data is deleted from the database due to the fact that the forum is adding rows rather than deleting them. I'm pretty certain that this won't be an issue on a forum of this size, but I'm happy to be proven incorrect if it fixes the problem.

Databases. You love em or you hate em.

Dead Eye · « **Reply #69 on:** 03 July 2013, 04:35:43 pm »

I'm not arguing, I find this sort of stuff fascinating believe or not and it's always good to bounce ideas off of people. I know for a fact that if I hadn't had the support of my fellow class mates at uni to throw ideas around, none of us would have done as well as we did in the end

Fragmentation can also occur when the disk drive is particularly full. Your are right though, most modern OS's and file systems will sensibly organise data and will periodically attempt to de-fragment what it can. I still think there is an issue with lots of small bits of data being collected from lots of different places on the hard drive though. If you think about it, information relating to the user is probably in one db table, then there is data about the post, the thread, avatars, logos, different scripts and includes, icons and so on. Its a big toll

However, even a crappy little netbook (the tiny 10" things with no optical drive) could probably run this without much issue. The problem is when you start tacking on lots of forums and other websites and resource intensive applications. The end result is that the poor hard drive takes one hell of a beating in terms of read / write requests where as data throughput is probably 10-20% of its maximum capacity.

As I was writing this, your second post has arrived (email alerts ftw). I was originally talking about file systems as fragmentation in the database is usually not significant for a forum - your are correct in that DELETE requests are a complete minority. I would suggest probably 80%+ is SELECT, the rest being UPDATE and INSERT a majority of which is probably UPDATE.

I don't think there is much that can be done to make the site faster since Farjo doesn't control the server and its setup - they rent hosting. It's just an interesting topic to discuss in a hypothetical world

Exupnut · « **Reply #70 on:** 03 July 2013, 04:50:37 pm »

Quote from: Dead Eye on 03 July 2013, 04:35:43 pm

It's just an interesting topic to discuss in a hypothetical world

No it isnt...its foccin borin. Yawn

Dead Eye · « **Reply #71 on:** 03 July 2013, 04:58:53 pm »

Only to the likes of you

Grahamm · « **Reply #72 on:** 03 July 2013, 06:17:32 pm »

Quote from: Exupnut on 03 July 2013, 04:50:37 pm

Quote from: Dead Eye on 03 July 2013, 04:35:43 pm
It's just an interesting topic to discuss in a hypothetical world
No it isnt...its foccin borin. Yawn

The "next topic" link is at the top and bottom of each thread...

simonm · « **Reply #73 on:** 03 July 2013, 10:31:33 pm »

Quote from: Dead Eye on 03 July 2013, 04:35:43 pm

If you think about it, information relating to the user is probably in one db table, then there is data about the post, the thread, avatars, logos, different scripts and includes, icons and so on. Its a big toll

I don't think there is much that can be done to make the site faster since Farjo doesn't control the server and its setup - they rent hosting. It's just an interesting topic to discuss in a hypothetical world

It might be http://www.ukhost4u.co.uk/shared-web-hosting/basic (I suspect the data is over a gig, but doubtful over 10 (mainly pictures I'd say). This gives you the ability to install SMF from a script and use mysql. In this case, although the admins wouldn't have control over the server's set up, I suspect they would have control over SMF's config but probably not ssh/root access. The awstats and webalyzer stats look like cool utilities though.

I'd guess the avatars, icons and pictures would be stored on disk as blob's in a database are generally bad news if you can serve them up in a simpler way you'd be advised to. If I'd written it I'd not store significant binary data in a database table if I could possibly help it. Think of the traffic flow http -> disk vs http -> database ->disk

Doddsie · « **Reply #74 on:** 03 July 2013, 10:47:33 pm »

Im with Exupnut on this one!!!!!!!