Handling sudden large traffic

[pilot830]

How are people handling a sudden traffic surge?

Example, a guy uploads a file, it goes on a file server, server1

he posts the URL somewhere, now 300 people are downloading the file.. maxing out server1 bandwidth

Now anyone who visits that link, is going to get slow download speeds.. because the bw is maxed out

So what are people doing to combat this?

 

[sukhman21]

do you allow full download speed for your visitors? maybe limit speed for users to 100 - 400Kbps depending on your network speed?

 

[enricodias4654]

Free users have limited speeds. All my servers have 1gbps links. It will require thousands of free users downloading at the same time to use 1gbps.

If you have enough premium users to use 1gbps, they will complete the download in few minutes.

 

[pilot830]

Free users have limited speeds. All my servers have 1gbps links. It will require thousands of free users downloading at the same time to use 1gbps.

If you have enough premium users to use 1gbps, they will complete the download in few minutes.

That might work out for you if you're not doing any streaming video, but majority of my traffic comes from streaming videos.. which the video/behind the video box is basically a download token url that has no speed limitation (thats just how yetishare works)

The streamable videos are 2000kbit so 400-500 users can easily max out one server that is on 1gbps.. and my post here is for situations where a file is getting clearly way more than 400-500 downloads. There just has to come a point and time where if a file is getting tens of thousands of streams, it cant possibly be hosted on one single file server

I have never seen here anyone discuss load balancing in terms of a video getting too many downloads, so it sounds like nobody has had to go through this yet

 

[pilot830]

do you allow full download speed for your visitors? maybe limit speed for users to 100 - 400Kbps depending on your network speed?

Doesn't help for video streaming, since the video loads at unlimited speed, which is required as well in order for the video to play properly

 

[enricodias4654]

There just has to come a point and time where if a file is getting tens of thousands of streams, it cant possibly be hosted on one single file server

I have never seen here anyone discuss load balancing in terms of a video getting too many downloads, so it sounds like nobody has had to go through this yet

I've made several suggestions regarding adding redundancy to the script by replicating files in 2 or more fileservers. This will solve your problem.

 

[pilot830]

There just has to come a point and time where if a file is getting tens of thousands of streams, it cant possibly be hosted on one single file server

I have never seen here anyone discuss load balancing in terms of a video getting too many downloads, so it sounds like nobody has had to go through this yet

I've made several suggestions regarding adding redundancy to the script by replicating files in 2 or more fileservers. This will solve your problem.

As long as the replication is only for the files that are actually getting downloads, and not every file even ones that are gettign no activity, then i'm fine with that..

 

[pilot830]

There just has to come a point and time where if a file is getting tens of thousands of streams, it cant possibly be hosted on one single file server

I have never seen here anyone discuss load balancing in terms of a video getting too many downloads, so it sounds like nobody has had to go through this yet

I've made several suggestions regarding adding redundancy to the script by replicating files in 2 or more fileservers. This will solve your problem.

Can you link to these suggestions? Are they in the RFC?

 

[pilot830]

While i'm at it, does anyone here get a lot of traffic for videos ? If so, how are you load balancing the traffic ? What are the procedures you are diong? As the script does not do any sort of load balancing.. and if you've got like hundreds of people trying to access a video it will easily stop your server

Or does no one here get much video traffic?

 

[enricodias4654]

My suggestions are in several posts in the RFC forum. I don't have the links right now.

The peak usage of one of my servers was 700 connections. The processor usage was in 50% (xeon, 6 cores) and 5gb of ram usage. Those 700 connections was using 750mbps.

 

[pilot830]

The peak usage of one of my servers was 700 connections. The processor usage was in 50% (xeon, 6 cores) and 5gb of ram usage. Those 700 connections was using 750mbps.

98% of my traffic is all streaming video, so this 700 connections thing is never going to work for me, since all my videos are converted to 2400-2500kbit and my servers are 1 gbps not 10gbps

 

[fatality43835]

What I can suggest is to get some extra servers and use DNS roundrobin along with rsync.
For example, you have 2 file servers (A and B), you get 2 other servers (A1 and B1). You setup DNS round-robin so that servers A+A1 uses the same subdomain, and servers B+B1 follows the same pattern. Then you setup rsync to sync all files/folders on server A to server A1, and then server B to server B1. In your YetiShare admin panel, you have servers A and B as normal, you do not add A1 or B1, as these are merely mirrors/clones of A and B. It should essentially work but keep in mind this was still a theoretical strategy for us as we weren't able to implement it in time.

Disclaimer: I did not get a chance to test this method as we moved our service off YetiShare and onto a custom solution for a various reasons.

 

[enricodias4654]

Replication and roundrobin dns will be overcomplicated and it may create several other issues with this script. I will probably not work using only rsync.

Some datacenters offer external storage boxes. You could order a large storage box and use it to store the files via ftp. You would need to modify your direct servers to fetch files from the storage box via ftp on the fly in order to have multiple servers accessing the same storage box. In this scenario a single storage box with 10gbps may be accessed by 10 servers with 1gbps each, assuming that you could get 10gbps from the disks.

Those storage boxes are usually expensive.

 

[enricodias4654]

I guess you can use storage box as network drives but be aware that the latency will be high.

 

[rfhmendes]

I believe the duplication of a specific file among several servers would be good enough if such functionality existed in the script. It would be much cheaper as well.