Based on their advice they would recommend capping the thread lengths to ~1000 posts
As far as I am aware there are only a handful of threads that are actually that long. I can only think of one in the 600 forum which is red98's stuttering thread. At the end of the day, software adjustments for a web forum aren't going to help that much when the traffic isn't that high.
(03-07-13, 12:13 PM)Dead Eye link Wrote: Based on their advice they would recommend capping the thread lengths to ~1000 posts
As far as I am aware there are only a handful of threads that are actually that long. I can only think of one in the 600 forum which is red98's stuttering thread. At the end of the day, software adjustments for a web forum aren't going to help that much when the traffic isn't that high.
Red's thread is "only" 935 replies long. Even capping the treads to 1000 means there's still room for another 64 replies; a number many threads don't even get near.
How about archiving threads where the last reply was over a year ago, making them read-only? Might that not lighten the load somewhat?
The Deef's apprentice
Not really
The load is soo low that this is more than likely a network / hardware problem more than anything else. In fact my bet is on the hard drive - its probably too slow to keep up with the number of requests from the SQL server (or equivalent - not sure what is used).
I partly rent a seedbox for torrents (all legal, I promise) but the hard drive is soooo slow that if you start more than 3 or so torrents then the system load rises to something like 10 - this will mean something to simonm more than anyone else I imagine. Essentially a system load of 1 means that everything is being processed as its being received, higher than 1 means that instructions are having to queue before they are dealt with. On most production servers, anything about 1 is usually bad  Where I work, anything above 0.2 is considered reasonably heavy load
How about starting a collection fund to upgrade the hard drives? I don't know how old the tech being used is...
The Deef's apprentice
That would never happen - its a corporate company who will basically make you upgrade your package on a monthly basis rather than deal with updating old hardware.
In any case, I spoke to Farjo about potentially moving the site but I think they are happy with where they are
(03-07-13, 12:13 PM)Dead Eye link Wrote: Based on their advice they would recommend capping the thread lengths to ~1000 posts
As far as I am aware there are only a handful of threads that are actually that long. I can only think of one in the 600 forum which is red98's stuttering thread. At the end of the day, software adjustments for a web forum aren't going to help that much when the traffic isn't that high.
I find that section a bit ambiguous to my understanding. I would have thought, that post size and content would be very relevant too. If every post had an image and some web links and those pictures were in different directories and the word censor had to work on every youtube url and 30 members were browsing a different thread I could see there being a large load on the disk I/O, especially if the database sits on the same drive. Even with a 10k HDD the read and write head would be flying all over the shop. Even with db and disk caching I can see performance being an issue.
Having the avatars in a single directory could help (if it isn't done already) http://wiki.simplemachines.org/smf/Perfo..._directory
as would having all the images in a single directory.
Obviously as I'm not an admin I can't tell if all of this is already done. I'm sure the admins have already done it (or it was probably standardised/bugfixed in a SMF release if it had that much of an impact) but without debug and logging it's hard to tell. If the ISP has moved machines to a higher spec machine (which I believe they did) then it could move the performance bottleneck somewhere else and turn it in to a sporadic or environmentally timed issue.
Not that it matters I think my 'help' isn't particularly wanted or needed so it's time to bow out 
Opinions are like A**holes, Everyone has one. Some people seem to have more than one though which is a bit odd.
Its more to do with where the data is stored on the hard drive (fragmentation)
Lots of tiny pieces of data (each post for example) requires a different read request if its not stored exactly next to another piece being requested (which is unlikely). Therefore you need an HDD with a very amount of IOPS more than transfer rates. This is where RAID arrays and SSDs come in to increase performance and yield higher IOPS to the OS / Applications
Its worth noting that having all the avatars in a single directory isn't going to make any difference to data fragmentation unless I'm missing something.
(03-07-13, 04:12 PM)Dead Eye link Wrote: Its more to do with where the data is stored on the hard drive (fragmentation)
Lots of tiny pieces of data (each post for example) requires a different read request if its not stored exactly next to another piece being requested (which is unlikely). Therefore you need an HDD with a very amount of IOPS more than transfer rates. This is where RAID arrays and SSDs come in to increase performance and yield higher IOPS to the OS / Applications
Its worth noting that having all the avatars in a single directory isn't going to make any difference to data fragmentation unless I'm missing something.
You only get fragmentation when you're deleting/overwriting data, if you're only adding it (which will mainly be the case on a forum) you shouldn't see much apart from possibly on the database files. On a unix box with ext3/4 you'll get virtually no fragmentation anyway. I don't believe fragmentation is much of a problem except in windows fat/fat32/NTFS environments. In conclusion I don't think fragmentation is an issue at all but we're all welcome to our opinions and I could be wrong.
Damn... I'm replying when I said I wouldn't. I could argue at length with you on the pros and cons of RAID0/5/6 etc but it's all environment/hardware specific so it'd all be guessing and when we don't know the set-up it's impossible and pointless to say. If a SMF forum needs a RAID array or SSD then you'd better be running the Ubuntu forums otherwise it's a hell of a spec machine just to run some php and mysql to post up some images.
Maybe you're right, maybe I'm wrong. Dunno.
Opinions are like A**holes, Everyone has one. Some people seem to have more than one though which is a bit odd.
I just figured out you're talking about database fragmentation rather than filesystem fragmentation.
Database fragmentation should really be hidden from the end user due to database/filesystem caching, the efficient use of stored procedures and the fact that very little data is deleted from the database due to the fact that the forum is adding rows rather than deleting them. I'm pretty certain that this won't be an issue on a forum of this size, but I'm happy to be proven incorrect if it fixes the problem.
Databases. You love em or you hate em.
Opinions are like A**holes, Everyone has one. Some people seem to have more than one though which is a bit odd.
I'm not arguing, I find this sort of stuff fascinating believe or not and it's always good to bounce ideas off of people. I know for a fact that if I hadn't had the support of my fellow class mates at uni to throw ideas around, none of us would have done as well as we did in the end 
Fragmentation can also occur when the disk drive is particularly full. Your are right though, most modern OS's and file systems will sensibly organise data and will periodically attempt to de-fragment what it can. I still think there is an issue with lots of small bits of data being collected from lots of different places on the hard drive though. If you think about it, information relating to the user is probably in one db table, then there is data about the post, the thread, avatars, logos, different scripts and includes, icons and so on. Its a big toll
However, even a crappy little netbook (the tiny 10" things with no optical drive) could probably run this without much issue. The problem is when you start tacking on lots of forums and other websites and resource intensive applications. The end result is that the poor hard drive takes one hell of a beating in terms of read / write requests where as data throughput is probably 10-20% of its maximum capacity.
As I was writing this, your second post has arrived (email alerts ftw). I was originally talking about file systems as fragmentation in the database is usually not significant for a forum - your are correct in that DELETE requests are a complete minority. I would suggest probably 80%+ is SELECT, the rest being UPDATE and INSERT a majority of which is probably UPDATE.
I don't think there is much that can be done to make the site faster since Farjo doesn't control the server and its setup - they rent hosting. It's just an interesting topic to discuss in a hypothetical world 
![[Image: 242673.png]](http://badges.fuelly.com/images/smallsig-uk/242673.png) ![[Image: 174802.png]](http://badges.fuelly.com/images/smallsig-uk/174802.png)
(03-07-13, 04:35 PM)Dead Eye link Wrote: It's just an interesting topic to discuss in a hypothetical world No it isnt...its foccin borin. Yawn
Just flapping about on this stagnant little pond on the outer rim of the internet.....yup.... :-))
Only to the likes of you
(03-07-13, 04:50 PM)Exupnut link Wrote: [quote author=Dead Eye link=topic=7522.msg85458#msg85458 date=1372865743]
It's just an interesting topic to discuss in a hypothetical world No it isnt...its foccin borin. Yawn
[/quote]
The "next topic" link is at the top and bottom of each thread...
03-07-13, 10:31 PM
(This post was last modified: 03-07-13, 10:32 PM by simonm.)
(03-07-13, 04:35 PM)Dead Eye link Wrote: If you think about it, information relating to the user is probably in one db table, then there is data about the post, the thread, avatars, logos, different scripts and includes, icons and so on. Its a big toll
I don't think there is much that can be done to make the site faster since Farjo doesn't control the server and its setup - they rent hosting. It's just an interesting topic to discuss in a hypothetical world
It might be http://www.ukhost4u.co.uk/shared-web-hosting/basic (I suspect the data is over a gig, but doubtful over 10 (mainly pictures I'd say). This gives you the ability to install SMF from a script and use mysql. In this case, although the admins wouldn't have control over the server's set up, I suspect they would have control over SMF's config but probably not ssh/root access. The awstats and webalyzer stats look like cool utilities though.
I'd guess the avatars, icons and pictures would be stored on disk as blob's in a database are generally bad news if you can serve them up in a simpler way you'd be advised to. If I'd written it I'd not store significant binary data in a database table if I could possibly help it. Think of the traffic flow http -> disk vs http -> database ->disk
Opinions are like A**holes, Everyone has one. Some people seem to have more than one though which is a bit odd.
Im with Exupnut on this one!!!!!!! :z
(03-07-13, 10:47 PM)Doddsie link Wrote: Im with Exupnut on this one!!!!!!! :z I.T. guys can be boring, we're a misunderstood lot (me more than most) . I'd recommend an unnotify click :rolleyes
Opinions are like A**holes, Everyone has one. Some people seem to have more than one though which is a bit odd.
Id recommend we stick to talking bikes!!!!!!
(03-07-13, 11:02 PM)Doddsie link Wrote: Id recommend we stick to talking bikes!!!!!! Bugger. I know nowt about bikes. Only had one for 6 months.
Opinions are like A**holes, Everyone has one. Some people seem to have more than one though which is a bit odd.
I find it interesting even though I don't know what half of it means lol.
(03-07-13, 10:31 PM)simonm link Wrote: [quote author=Dead Eye link=topic=7522.msg85458#msg85458 date=1372865743]
If you think about it, information relating to the user is probably in one db table, then there is data about the post, the thread, avatars, logos, different scripts and includes, icons and so on. Its a big toll
I don't think there is much that can be done to make the site faster since Farjo doesn't control the server and its setup - they rent hosting. It's just an interesting topic to discuss in a hypothetical world
It might be http://www.ukhost4u.co.uk/shared-web-hosting/basic (I suspect the data is over a gig, but doubtful over 10 (mainly pictures I'd say). This gives you the ability to install SMF from a script and use mysql. In this case, although the admins wouldn't have control over the server's set up, I suspect they would have control over SMF's config but probably not ssh/root access. The awstats and webalyzer stats look like cool utilities though.
I'd guess the avatars, icons and pictures would be stored on disk as blob's in a database are generally bad news if you can serve them up in a simpler way you'd be advised to. If I'd written it I'd not store significant binary data in a database table if I could possibly help it. Think of the traffic flow http -> disk vs http -> database ->disk
[/quote]
From information that Farjo has given me, I believe they are on a higher package - the site is growing quickly and will soon pass the 10GB disk limit
I would find it unlikely for the avatars to be stored as blobs but could be wrong - seems a very strange approach. In any case, UKHost4U have been described to me as "ok". Their support seems ok based on what the admins have mentioned but I expect they are overloading their servers with users for monetary gain which is the cause of the slowness. Farjo mentioned about a New Server in the General section - I guess they just moved the account which is a fairly straight forward process and hope that the new one has less intensive users. Its hit and miss at the end of the day
|
