You can't delete the admin folder, cron tasks are inside it. Renaming it will just make the attacker take a little longer to figure out the new name. The attacker will assume that a strange login screen is the admin.
As I have fixed ips on my company I restricted the access using .htaccess, but...