File hash with md5_file too slow for large files

[faridadobani5832]

Hello,

I am getting this error in my error_log file

PHP Fatal error:  Maximum execution time of 180 seconds exceeded in /core/includes/uploader.class.php on line 920

This is line 920's code

// create file hash
$fileHash = md5_file($tmpFile);

Whenever I upload a large file (1GB+) my server load increases. and I think this is the issue. Can I use

exec('md5sum '.$filename);

Is there any other solution?

I may be wrong please correct me.

 

[enricodias4654]

All my servers can calculate md5 of large files in few seconds. Which processor are you using?

 

[faridadobani5832]

Processor: Intel Xeon E3-1245
RAM: 4x DDR3 8192 MB
Server: Apache
OS: CentOS 7.2

Moreover I may be wrong with the md5 problem. There may be something else causing the server load. I guess its md5 because of the max execution time error. I also have file with 25GB size so those files may bea giving server load while md5 hashing.

Please share your thoughts on this.

 

[enricodias4654]

It's a nice config. I don't have 25gb file here but it took 10 seconds to calculate the md5 for a 8gb file.

Try to monitor your server with htop and iotop while uploading the file. It will provide you more information about what is happening.

 

[wrestletube5692]

don't use Apache server try to move to php-fp + Ngnix I have the following configuration and its take few seconds to calculate md5 hash

CPU: 2 x E5-2440
Ram: 64GB
php-fpm+ Ngnix and SQL server
my users are uploading large files like 4GB, 8GB and above and I never get high server load never during uploading or in downloading. I just get high load during ffmpeg process. which is normal

my suggestion is to configure your server with Ngnix as Apache servers are not good enough for high traffic

 

[enricodias4654]

don't use Apache server try to move to php-fp + Ngnix I have the following configuration and its take few seconds to calculate md5 hash

CPU: 2 x E5-2440
Ram: 64GB
php-fpm+ Ngnix and SQL server
my users are uploading large files like 4GB, 8GB and above and I never get high server load never during uploading or in downloading. I just get high load during ffmpeg process. which is normal

my suggestion is to configure your server with Ngnix as Apache servers are not good enough for high traffic

Php running as an apache module is 2x faster than php-fpm. I don't this this will solve his problem.