Scaling Super-Thread

[paypal1352]

I want to start this thread if I may as a central location for scaling advice from our more server-savvy members.

What are your recommendations/experiences with making yetishare scalable, what's your server setup and how do you ensure that your setup positively deals with surges of traffic and a lot of simultaneous traffic in general?

 

[enricodias4654]

What are your recommendations/experiences with making yetishare scalable

Work on the code, a lot. I took 1 year working on it before start using. Me and 2 other programmers.

what's your server setup

Several servers with different configs. From 2x 2tb with 16gb ram to 15x 6tb with 128gb ram. All with hardware raid controllers with write cache, bbu and ecc memory.

how do you ensure that your setup positively deals with surges of traffic and a lot of simultaneous traffic in general?

The only bottleneck is the link speed or a single server, usually 1gbps. There is not much you can do about it. I didn't implemented any load balancing yet, didn't needed it. The files are distributed across many servers.

 

[sukhman21]

I have been using multiple VPS's (1GBPS bandwidth each) from different companies as my fileservers and then a backup server to which all fileservers rsync's files to incase of fileserver downtime.
For the core script, i have a cloud VPS with 64GB of RAM. using cache for files and SQL queries.
Then i have my own CDN (using nginx and varnish) for css and js files. oh BTW shrink down your css and js. remove what you don't use. the script comes with too much extra files, i was able to shrink it down to more than half the size.

So far website is running fine. i ran about 500 virtual users on the site and it holded up. the load time increased to 4 seconds from 2 but it was still working.

@enricodias4654: the code you are talking about - Is it for load balancing or for pointing users to download files from specific fileserver based on the available bandwidth / speed / current downloads per fileserver, etc ?

 

[enricodias4654]

I have been using multiple VPS's (1GBPS bandwidth each) from different companies as my fileservers

Then you don't have 1gbps on each. This 1gbps is shared to all customers in this vps.

and then a backup server to which all fileservers rsync's files to incase of fileserver downtime.

I don't see the need in backups on a small/medium site. Raid 10 or raid 6 will keep your data safe just fine.

For the core script, i have a cloud VPS with 64GB of RAM. using cache for files and SQL queries.

Those 64gb ram are probably shared by several users. A server with 8gb ram should handle the main site and database just fine if you have less than 50 requests/sec.

Then i have my own CDN (using nginx and varnish) for css and js files. oh BTW shrink down your css and js. remove what you don't use. the script comes with too much extra files, i was able to shrink it down to more than half the size.

A cdn for just js and css? Do you really mean cdn?
My js and css are compacted together in a single file, compressed in .gz. It is 10% of the original size.

@enricodias4654: the code you are talking about - Is it for load balancing or for pointing users to download files from specific fileserver based on the available bandwidth / speed / current downloads per fileserver, etc ?

When I said to work on the code, I meant the whole code. I didn't code anything regarding load balancing and file redundancy, I didn't needed to do it yet.

 

[pilot830]

Load balancing, script doese't support it, so you've got to build your own solution

 

[pilot830]

I don't see the need in backups on a small/medium site. Raid 10 or raid 6 will keep your data safe just fine.

There are any number of unforseen disasters that could happen. Having off site backup is important to some of us

 

[sukhman21]

@enricodias4654 - you are right, the VPS 1GBPS is shared but i haven't had any slowness issues so far touchwood!!

As for backups, for some of my clients, i allow them storing their business / personal files so i need a backup in my case incase something happens. and hence the backup server incase they need to access their files incase my fileserver goes down.

my cloud VPS was the first server i started my website on and that time i was hosting all the files on the same server. i just never changed it because i had paid it for 2 years in the beginning. it is coming up for renewal at the end of this year so yes that was the plan - to go to smaller cloud VPS. You are right, max RAM used i have seen for my main site is 6GB. because i have 64, i gave a lot of RAM to SQL queries caching.

For CDN, i did reduce the number of CSS and JS files and my current page size is 436KB (as per pingdom). i will look into merging them all to 1 file. i have 5 JS and 2 CSS right now.

Thanks for your comments and suggestions.

 

[enricodias4654]

Having an offsite backup doubles your costs. If you really need to ensure the safety of the files, you should fix the bugs on the script first. There are bugs that may end up deleting files, such as when you are moving files from one fileserver to another and the transfer or the mysql query fails.

About CDN, cdn means several servers in several locations to deliver static content with lower ping. This includes images and not only js and css.

About mysql cache; most of the queries in this script don't use cache. There are some pre requisites in order to use cache in the mysql, such as the rows and field sizes. Even if the mysql uses cache for everything you won't use 64gb, unless you have a huge database. And if a dc sells a vps with 64gb ram, it's probably overselling.

I have 6 servers with 1gbps each, over 120tb in files. All servers are using less than 4gb ram. The main server is using less than 8gb for the mysql and apache processes. The script serves files with php only and the cpu usage on all fileservers is always less than 10%.

 

[sukhman21]

i see, and yea i got a lot less files than 120TB, that is quite a lot actually which is why you are not keeping the offsite backup (due to costs and its understandable). meaning there is a lot of work to be done in optimizing the script. i am really no developer, just know a little bit coding but that's about it. and yea once my contract expires for VPS, i ll move over to cheaper and one with less RAM. thanks again.

 

[pilot830]

Having an offsite backup doubles your costs. If you really need to ensure the safety of the files, you should fix the bugs on the script first. There are bugs that may end up deleting files, such as when you are moving files from one fileserver to another and the transfer or the mysql query fails.

Any serious online business is going to have offsite backups of some kind. Anything short of that is irresponsible and a narrow view of their own business. Having this kind of attitude that you just need to "bug fix" everything and thats going to avoid you any unforseen disaster situation that requires you to pull a copy of your data from somewhere offsite. Just because you've been lucky and haven't had anything happen to you yet doesn't mean offsite backups arent important, but its your choice.

 

[enricodias4654]

Any serious online business is going to have offsite backups of some kind. Anything short of that is irresponsible and a narrow view of their own business. Having this kind of attitude that you just need to "bug fix" everything and thats going to avoid you any unforseen disaster situation that requires you to pull a copy of your data from somewhere offsite. Just because you've been lucky and haven't had anything happen to you yet doesn't mean offsite backups arent important, but its your choice.

I'm not saying they are not important, I'm just saying that there are more important things to fix before investing in backups. With the current script you can lose files randomly and don't even know it, which is far worse. Actually, implementing file redundancy (save the same file in multiple servers) will avoid the need for backups.

With my current setup there is just not enough money to use backups or to implement file redundancy.

 

[pilot830]

I feel like there's lots missing from this script, compared to a rival script, I hope more things are going to be added

 

[enricodias4654]

I feel like there's lots missing from this script, compared to a rival script, I hope more things are going to be added

Do you mean the xfilesharing? It's old and the code is a mess. Yetishare is still better than it.

 

[pilot830]

I feel like there's lots missing from this script, compared to a rival script, I hope more things are going to be added

Do you mean the xfilesharing? It's old and the code is a mess. Yetishare is still better than it.

well im all team yetishare, so i didn't want to mention anything by name.. i only said that becuase as mentioned in this thread, theres many things missing from the script. i see how on the other scripts, they seem to have some things yetishare doesnt, such as

- for the rewards, they have this already built into the script or plugin for rewards http://i.imgur.com/D2IzEE1.png the whole reward mode stuff..
with yetishare, someone would have to custom code it, or id have to go and pay someone to do it...

- how can i do this in yetishare?
"Downloads/Streams are counted up to 3 times every 24 hours per IP." -- doesnt appear possible
"Rewards arent counted strictly per view. Its depending on the conversion of advertisements on player ADS" -- not possible
"Any non browser downloads are counted by factor 0.1" -- doesnta ppear possible

I dont expect all of that to be in the script, im sure some of this is just way too custom, but would be nice to seee more stuff in the script

my biggest gripes at the moment are zero load balancing within the script, and the whole video buffering issue:

- on other scripts, when you load a video using chrome, the video buffers as you play, and or if you pause, it continues to buffer. this is important for people who have slower connections or are far away from the server.. so it helps that the video buffers some, so they can pause, let it buffer some, then play and watch whats already buffered while the rest continues to buffer. right now when you view a video on a site thats powered by yetishare using chrome , the video does not buffer, only a small chunk ahead of where you are, that's it.. im not entirely sure why this is, either somethign to do with the headers, or the fact that the other scripts possibly (not 100% sure), load the mp4 right off the web server.. in a dir.. like this site, thevideo.me , they use xfs, and i see on the source code for their video, it says something like

{"file":"https://d8584.thevideo.me:8777/osjta56uz2oammfvg6rveg4qou5q7j6p63y6taujhpalrh4x52p6fwj7xesj26tppnoy5lsbxsevi7jnb5in4lazol3dcjcavatqsoxh44glwyhvt64boauqrtebf6cfu32goj56tyw5cguvomiybx6oturo5m5fyhh32bibmzomfawfekmklxz3vhjihvicqa3la5bdiix56trz25izcdfdh4dtvpe3v6negsfpmzboraq2mllfy2k652melsst7zhey7ucpy3qa2gsaznicysgenqa/v.mp4","label":"360p"}]


so it appears that they load the video out of a directory, that has random hash or randomization to it.. and then the v.mp4 is the actual file.. so i think by loading the video off the web server, the video buffers.

but with yetishare, its loaded not off the webserver, the file is hidden in the files/localfilepath directory, through a download token URL

last i checked you said that you dont use yetishare for video, so you may be unaware of these video related issues

I haven't heard back from yetishare about this issue. this is not a jwplayer or videojs issue. I've now started the process of hiring a guy to fix the problem, hopefully he does, then ill be all set.. What sucks is, I would love to rather pay adam to do this.. and then he could even incorporate the fix into the script for everyone else.. im fine with that.. But i guess hes too busy or something.. which makes me feel like, he should just hire people, then he can afford to take on custom jobs like i just said.. and he could make money from that, and everyone benefits. but maybe its all easier said than done


i have an idea with the video thing though, im getting more into php, but not knowledgable enough to fix anything major, hence why i hired someone to do it.. but for fun i think i might try

- in the media player ,most likely _append file download bottom.php, get the downlaod token value
- use php to create a directory in public_html/ and the name of the DIR will be the download token
- then in that directory, use php to symlink the file into that DIR,
- then change the download media URL value (where jwplayer is, for the file, to a url that is http://file.server/[download_token_dir_name]/file.mp4

that might work and things would work the same

Only thing id have to figure out then is, when the downlaod token would expire as per usual, id have to get it so that it removes that directory.. Just think i might try this for fun to get more experience in php

 

[enricodias4654]

I don't use yetishare for video so I don't know how it was coded. I do know that the url doesn't give you much information. Just because it looks like a folder with a file, it doesn't mean it is. There are several ways to stream a video, the most used (i think) is to send several small files in sequence, just like youtube and some porn sites used to do. There are several advantages in this method.

Yetishare probably sends the video the same way it sends an ordinary file.

About load balancing, no other script offers it, as far as I know.

 

[sukhman21]

not sure if this is a traffic or bandwidth thing but i have not seen this issue on my website with videos. It does buffer fine for me and i am able to play video from the middle once it buffers and it plays right away (don't keep buffering). I know this was an issue with chrome before but i seem to not have any more issues with chrome either.
Before taking this screenshot, i tested video buffering on IE, firefox and chrome.

For more features, i agree Adam and team should hire another person for custom dev work. it is hard to find someone on the outside because they don't know how the core script is programmed. Took me long time before i could find someone to develop API for my script.

 

[rexcarnation5449]

You right, on my website it's doesn't buffers too.

It's only buffers few seconds upward and it stop until it reached the buffers point, and get buffers again.

I am using videojs as a player btw

 

[pilot830]

i am able to play video from the middle once it buffers and it plays right away (don't keep buffering)

I think you may have misundrestood the issue. Load up the video using chrome, hit play, and then hit pause, and you will see that the video does not buffer beyond a tiny chunk

The screen shot you have provided is for firefox. As I said, the video buffers and continues to buffer on firefox, but not on chrome. Please show a screen shot of chrome, showing that the video is buffering beyond a tiny chunk. As you can see here (and ive already provided this screen shot), the video does not buffer beyond a tiny chunk when played with google chrome

 

[sukhman21]

ahh i see what it is. so when we pause the video it only buffers that chunk and stops. when you play it again, it will continue to buffer again..... have you seen this ?

 

[pilot830]

ahh i see what it is. so when we pause the video it only buffers that chunk and stops. when you play it again, it will continue to buffer again..... have you seen this ?

Yes on google chrome it will continue to buffer again, but only a tiny chunk.. If you just play the video and leave it, you will see that as you play the video, it continues to buffer but only a small margin... Never more than a small margin.. If you pause, buffering stops altogether..

On firefox though, it will buffer LARGE margin while playing, and keep giong....... Or if you pause, it will buffer the entire video if you wait for it.. this is what i need.. but i need this on chrome.. so i wish yetishare would respond or realize that the problem has to do with the script code, but it seems they are too busy i guess.. which is why i wish they'd hire more help.. but even if they had help, maybe they wouldn't consider this buffering thing a problem OR a priority.. so i guess i have to go find someone to hire, but i dont know where i can find someone that is reliable and wont charge me $1000 either.

It seems nobody on here gets a lot of video traffic..... so i guess thats why nobody has complained, I do get video traffic. Right now im seeing 622 downloads , all video.