Download take a long time to start

P

parham4229

New Member
Sep 3, 2014
46
1
0
Hi

We moved all of our users from XFS to yetishare few days ago (3-4 days ago), I been noticing an issue which get worse and worse as my users starting to use the new system (We have a lot of users, to be more specific about 50K), I know my servers are powerful enough to handle this traffic because we were doing so easy when using XFS but right now the servers which have a bit of load take about 40 seconds to think about downloading the file and user get bad gateway after that, the other servers which are functioning normally it take about 30 seconds for the download to start which again is not normal at all.

now here are some information, I did check the server when it was returning bad gateway and there was about 200 php process running but I can't see exactly what they were (this is one of the file servers)

we have enabled the x-axcel mod in nginx and did the configuration as we were asked but no luck, here is my nginx configuration

Code: 
server {
    listen       80;
    server_name  serverurl.domain.com;
    root /home/path/public_html;
    index index.php;
    client_max_body_size 50G;

    location ~ \.php$ {
        if (!-e $request_filename) { rewrite ^/(.*) /index.php?_page_url=$1 last; }
        fastcgi_pass   unix:/var/run/php5-fpm.sock;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME   $document_root$fastcgi_script_name;
	fastcgi_param  MOD_X_ACCEL_REDIRECT_ENABLED on;
        include        fastcgi_params;
    }

    location / {
		if (!-e $request_filename) {
			rewrite ^/(.*) /index.php?_page_url=$1 last;
		}
    }

    location /files/ {
        root /home/path/public_html;
        post_action @afterdownload;
        internal;
    }

    location @afterdownload {
        proxy_pass http://myserverip/plugins/rewards/site/_log_download.php?request_uri=$request_uri&remote_addr=$remote_addr&body_bytes_sent=$body_bytes_sent&status=$request_completion&content_length=$content_length&http_user_agent=$http_user_agent&http_referer=$http_referer&args=$args;
        internal;
    }

    location /core/logs/ {
        deny all;
    }
}
Any idea where the issue might be? if the system could generate symlink like XFS did and provide direct link wouldn't that be easier instead of all these hassle?

Regards
Parham
 
P

parham4229

New Member
Sep 3, 2014
46
1
0
Looking around the script and also nginx features I found this

http://wiki.nginx.org/HttpSecureDownload

it pretty much does the same thing as what I think is done in yetishare but more efficient so there is no php load to load the file and also no delay I think, has anyone changed the script around to use this feature of nginx instead of how the links are originally generated and handled in yetishare

Regards
Parham
 
C

coffee

New Member
Jul 23, 2014
83
0
0
Have you enabled X-Accel-Redirect for nginx? I really know very little about nginx, so I can't be of much assistance.

https://support.mfscripts.com/p/kb_view/2/
 
P

parham4229

New Member
Sep 3, 2014
46
1
0
Yes, I looked around the nginx but I can't find any issues, it seems to be more an issue of the download proccess and how it works, for a large website it's not very ideal to be honest, I'm still doing some re-search but I might rewrite the download proccess base on the nginx function that I mentioned earlier
 
P

parham4229

New Member
Sep 3, 2014
46
1
0
@xperience024227, we had most of the plugins of XFS and we used them for about 2-3 year BUT I never ever liked them, they do not provide %100 of source code and there are some bugs in the script which they are not willing to sort out because they don't accept that it's there, at least with this script even if I don't get support I can get my staff to fix the issues (but that's not even the case, most of the tickets get respond in a good time manner I would say)

And about this issue we are nearly done re-writing the download process, when I finish I might make it available to purchase as a plugin if you want to make your user downloads a lot easier and faster (getting PHP involved in download is just wrong, maybe small websites would be ok but a large website like ours it's just a NO NO :d )
 
  • Like
Reactions: parsiupload
P

parham4229

New Member
Sep 3, 2014
46
1
0
@thefilebay using PHP is just wrong in large scale for downloads, we coded the download system ourselves now and patched the codes few hours ago, the servers are stable again and the speed is blazing fast ;)
 
adam

adam

Administrator
Staff member
Dec 5, 2009
2,046
108
63
parham4229 said:
@thefilebay using PHP is just wrong in large scale for downloads, we coded the download system ourselves now and patched the codes few hours ago, the servers are stable again and the speed is blazing fast ;)
Click to expand...
It doesn't use PHP. If you use the option XAccelRedirect it hands off onto Nginx (assuming the nginx settings are working). You still need PHP at the start of the process for validation, security etc but this adds little overhead.

Good to hear you are happy with access to the source code :)
 
adam

adam

Administrator
Staff member
Dec 5, 2009
2,046
108
63
By the way I couldn't see anything really wrong with your nginx config. "location /files/" we have as "location /files" but I don't think that would make a difference. I assume you restarted nginx on all servers and enabled the option in site settings, admin?

HttpSecureDownload also looks interesting, I don't see it as a replacement for XAccelRedirect but another choice. You'll still need PHP at the start to generate the secure url. Also double check you can still use the download complete callback for rewards. I'd be interested to know how you get on.
 
P

parham4229

New Member
Sep 3, 2014
46
1
0
We managed to get HttpSecureDownload working and also the PPD is working but I haven't got a change to work on full download yet, I would update when I get that working as well :)
 
G

green3ye

New Member
YetiShare User
Wurlie User
Dec 1, 2012
3
0
0
I am having the same issue with XAccelRedirect. I was able to plug it in nginx settings and make it work but take like 5 -10 seconds for download to start. I have enabled it from the admin side. I restarted both nginx and php-fpm on both server and file server side. Is there something we are missing thats causing the slow download start up?
 
Netnaija

Netnaija

New Member
YetiShare User
Apr 3, 2021
20
2
3
parham4229 said:
Hi

We moved all of our users from XFS to yetishare few days ago (3-4 days ago), I been noticing an issue which get worse and worse as my users starting to use the new system (We have a lot of users, to be more specific about 50K), I know my servers are powerful enough to handle this traffic because we were doing so easy when using XFS but right now the servers which have a bit of load take about 40 seconds to think about downloading the file and user get bad gateway after that, the other servers which are functioning normally it take about 30 seconds for the download to start which again is not normal at all.

now here are some information, I did check the server when it was returning bad gateway and there was about 200 php process running but I can't see exactly what they were (this is one of the file servers)

we have enabled the x-axcel mod in nginx and did the configuration as we were asked but no luck, here is my nginx configuration

Code: 
server {
    listen       80;
    server_name  serverurl.domain.com;
    root /home/path/public_html;
    index index.php;
    client_max_body_size 50G;

    location ~ \.php$ {
        if (!-e $request_filename) { rewrite ^/(.*) /index.php?_page_url=$1 last; }
        fastcgi_pass   unix:/var/run/php5-fpm.sock;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME   $document_root$fastcgi_script_name;
    fastcgi_param  MOD_X_ACCEL_REDIRECT_ENABLED on;
        include        fastcgi_params;
    }

    location / {
        if (!-e $request_filename) {
            rewrite ^/(.*) /index.php?_page_url=$1 last;
        }
    }

    location /files/ {
        root /home/path/public_html;
        post_action @afterdownload;
        internal;
    }

    location @afterdownload {
        proxy_pass http://myserverip/plugins/rewards/site/_log_download.php?request_uri=$request_uri&remote_addr=$remote_addr&body_bytes_sent=$body_bytes_sent&status=$request_completion&content_length=$content_length&http_user_agent=$http_user_agent&http_referer=$http_referer&args=$args;
        internal;
    }

    location /core/logs/ {
        deny all;
    }
}
Any idea where the issue might be? if the system could generate symlink like XFS did and provide direct link wouldn't that be easier instead of all these hassle?

Regards
Parham
Click to expand...
Can your team help me with my Nginx setup
 
You must log in or register to reply here.
Top Bottom