Support > About independent server > Troubleshooting methods for website servers being infected with malware, causing hard drives to fill up quickly.
Troubleshooting methods for website servers being infected with malware, causing hard drives to fill up quickly.
Time : 2025-11-19 14:56:26
Edit : Jtti

  Drive-by download attacks not only consume a large amount of system resources but can also become a springboard for further intrusions and data breaches. The rapid filling of hard drives typically manifests as abnormal website access, continuously increasing logs, bloated application cache, and escalating server load, even leading to high disk I/O, database write failures, and abnormal service interruptions. If this situation is not quickly identified and resolved, it can not only impact business operations but also seriously damage website reputation and user data security. Therefore, mastering systematic and professional troubleshooting methods is crucial for operations and maintenance personnel.

  First, identifying the symptoms and scope of the rapid increase in server hard drive usage is the prerequisite for troubleshooting. Checking disk usage via commands is the most basic method, for example:

df -h

  This command displays the used space, available space, and mount path for each mount point, helping to quickly locate abnormal partitions. Combining it with the `du` command allows for further analysis of the usage of specific directories.

du -sh /* 2>/dev/null

  If you find that the space in a directory is abnormally increasing, such as /www/wwwroot/, /tmp, or /var/log, it indicates that a malicious script may be continuously writing files to that path. Since drive-by download attacks typically generate a large number of temporary files, logs, or cache files, quickly identifying the directory with the fastest growth is crucial for subsequent analysis.

  After confirming the abnormal directory, you need to further locate the files that are consuming the most space. You can use the following command to list the top ten largest files:

find /www/wwwroot -type f -exec du -h {} + | sort -rh | head -n 10

  By checking file size, creation time, and modification time, you can determine if a file was recently created. If you find a large number of files with suspicious, irregular names, containing random characters, or disguised as system files, you can almost certainly conclude that these files were generated by malware. Furthermore, malicious scripts are often stored in formats such as PHP, JS, GIF, TXT, and LOG, and sometimes exist as hidden files (starting with a dot). Therefore, you need to add the `-name ".*"` option to check for hidden files.

find /www/wwwroot -type f -name ".*" -exec ls -lh {} \;

  By comparing the file content with its hash value, it's possible to further determine if the file contains known malware. A common method is to examine the file header for typical malicious code signatures such as PHP eval, base64_decode, gzinflate, and str_rot13. For example:

grep -rE "eval|base64_decode|gzinflate|str_rot13" /www/wwwroot

  This operation can quickly locate files containing malicious executable code.

  Besides static file analysis, running processes may also continuously generate junk files or write logs, causing the disk to quickly fill up. You can use `lsof` to view files that are open but still occupying disk space:

lsof | grep deleted

  These types of files have usually been deleted, but the process still retains the file handle, so the disk space is not reclaimed. Once the corresponding process is located, restarting or killing the process will free up the space. For example, PHP-FPM, Nginx Worker, and Node.js services may exhibit this behavior.

  For web servers, common paths for malware attacks include /www/wwwroot/, upload directories, cache directories, and temporary directories. Therefore, checking write permissions, the number of files, and file types in these directories is crucial. Using `find` combined with modification time can filter files generated within the last 24 hours.

find /www/wwwroot -type f -mtime -1

  Analyzing file types and size trends can provide a preliminary assessment of whether the attack is ongoing and whether the files are generated in batches by automated scripts.

  Logs are also a significant source of rapid disk depletion, especially when malicious code injection causes frequent abnormal requests that trigger error logs. For example, Nginx, Apache, and PHP error logs can balloon from tens of MB to tens of GB in a short period. The following command can be used to check log space usage:

du -sh /www/wwwlogs/*

  Once an abnormal log file is detected, it can be temporarily cleaned up or compressed, while simultaneously identifying the source of the logs. For example, by checking Nginx's access.log records, the IP address or URI with the highest frequency of abnormal requests can be identified.

awk '{print $1}' /www/wwwlogs/access.log | sort | uniq -c | sort -nr | head -n 20

  Analyzing the source of abnormal access can block the continuous execution of malicious scripts.

  In addition to file and log analysis, the database also needs to be checked. Malicious scripts often write large amounts of useless data through SQL injection or remote execution, causing database files to swell rapidly, especially MySQL's ibdata1, binlog, or database cache directories. If abnormal database file usage is detected, the number of table records and table size can be checked using MySQL queries.

SELECT table_schema, table_name, data_length+index_length 
FROM information_schema.tables 
ORDER BY data_length+index_length DESC 
LIMIT 20;

  After confirming the abnormal tables, you can back up important data, clean or optimize the tablespace, and check the application for vulnerabilities that could lead to malware injection.

  After checking files, logs, processes, and databases, abnormal access may also be present at the network layer. For example, malicious scripts might send a large number of HTTP requests or external data in the background, causing cache files and log files to swell. You can use `netstat -ntlp` or `ss -ntlp` to check the current listening ports and connection counts to identify the source of abnormal traffic.

netstat -ntlp
ss -s

  Combining abnormal IP blacklists or firewall rules can effectively block attack sources.

  After completing the above investigation, it is essential to clean up malicious files and restore the system environment. Recommended procedures include: backing up important files and databases, completely deleting malicious files, cleaning up occupied space, rebuilding the cache directory, and restarting critical services. To prevent similar attacks from recurring, server security measures need to be strengthened, including disabling unnecessary write permissions, restricting upload directory permissions, regularly scanning web files, deploying a Web Application Firewall (WAF), upgrading application and plugin versions, and setting up automatic log rotation and disk alerts.

  For long-term maintenance, scripts can be used to periodically scan the website directory for files with characteristics such as `eval` and `base64_decode`.

grep -r --include=*.php "eval(base64_decode" /www/wwwroot/

  In conjunction with inode, file size, and modification time, alerts or automatic isolation mechanisms should be generated. Simultaneously, important data should be backed up regularly, and disk monitoring thresholds should be set to prevent business interruptions caused by rapid disk fullness.

  Overall, the problem of website server malware infection leading to rapid disk fullness is highly insidious and destructive, requiring investigation from multiple dimensions including the file system, logs, processes, databases, and network. Through systematic analysis and professional methods, not only can malicious files and abnormal usage be quickly located, but server operation can also be restored, and defense mechanisms can be established to prevent similar problems from recurring. When facing this problem, operations and maintenance personnel should fully understand the attack methods, disk usage mechanisms, and file system management methods to form a complete investigation process. Only in this way can they effectively deal with malware attacks and the resulting risk of disk fullness while ensuring business continuity.

Relevant contents

Analysis of the Necessity of Configuring Redundant Servers and Backup Systems in Large Enterprises US Server CPU Performance Evaluation Standards and Technical Specifications Guide to Cleaning Up Website Access Logs on BT Panel Reasons and solutions for Linux partition unmounting failure SSD vs. HDD: Which to Choose? Analysis of the Real Difference in Game Loading Speed ​​Between the Two Configuration requirements and performance optimization for setting up a Southeast Asian store cluster using a Singapore server. How to achieve end-to-end encryption protection for game servers? Why are the upload and download speeds asymmetrical on Hong Kong's high-bandwidth servers? Architecture and Practical Applications of the Singapore Server Bus Specification (QPI) A Practical Guide to Server Redundancy in Japan: From Infrastructure to Optimization Strategies
Go back

24/7/365 support.We work when you work

Support