January 21, 2018

The strange launch of Hackers Chat

I recently launched Hackers Chat as an experiment to build the community site that I wish existed : public chat room based communites, pseudonymous users, transparent moderation logs, an open source code base and a site that is welcoming to non-technical users. I launched it as a single chat room for all discussions and support for user created chat rooms will be added shortly. An analogy might help : Reddit for chat rooms.

I originally launched the site under the name Bored Hackers , on the 1st of January, 2018 to an underwhelming response. The site didn’t get traction in any of the communities where I posted, yet what followed was an interesting series of events. Someone noticed that the chat room didn’t have basic anti-spam measures and posted a link to 4chan asking for help in spamming the chat room. Soon, the trolls and spammers showed up and started posting links to porn, messages about Hitler and other offensive content.

It didn’t take long for the conversation to go downhill since they succeeded in provoking most of us. I started manually deleting the troll accounts but it didn’t help much since creating an account was as simple as entering a username, email id and password. There was no email verification or any rate limiting. They created new accounts, started the conversation with “You ban me [insert cuss word]” and continued doing what they do best.

While I was playing Whac-A-Mole with the trolls, something else happened, they started using scripts to automate spamming. The Bee Movie script is popular with trolls for some reason and the chat room was suddenly filled with thousands of spam messages and it was not possible to keep track of the messages legitimate users were sending. What didn’t help was that my javascript scrolled to the bottom everytime there was a new chat message (fixed now) so it wasn’t possible to scroll and read the chat history. After sending a few messages informing users to close this chat room and come back in a few hours once I fixed the situation, it was acknowledged by most of the users. But no one closed the chat room since it was fun to watch the train wreck in real time.

The spamming, trolling and Whac-A-Mole continued for another hour before I closed registration the dinosaur way : ssh’ed into the server, edited the code, killed the process running the application and started another process using nohup python manage.py runserver &. I know, I know, I should never do this, but I guess…MVP. I’ve since switched to running daphne, a worker process and celery using systemd.

Since it was launch day, there were quite a few users coming in every hour and idling for a few minutes. Unfortunately most of the legitimate users could not signup since I had closed registration. Soon, the normal conversations resumed and we discussed a lot of interesting topics ranging from moderation to 4chan to startups and cryptocurrencies. You can read a part of it here. I interrupted these conversation periodically to inform the people watching the chat to send me an email if they want to join the conversation. I ended up receiving 7 emails asking for an invite. I’m guessing the trolls cost me quite a few missed signups. The chat room has presence, which shows how many users are connected to the chat room. At any given point over the 3 days after launching, there were 15 to 20 people connected to the chat room.

Registration remained closed and the conversations between users who already had an account continued. On the 6th of January, another spamming incident occured! A troll with the username “arnold” had an account before I closed registration. When no one was connected to the chat room, he decided to whip out his spamming script and sent over 5000 messages interspersed with messages such as “Oh this chat room hasn’t crashed”, “You’re still up”. Well, 5700 messages over 30 minutes is not even close to a DoS attack. Once I noticed what was happening, I promptly deleted his account along with all the spam messages.

A few minutes after this incident, I received an email from Google Cloud informing me that my account had been suspended because of billing related issues! Can the day get any better? It would take a few days to get the situation sorted so I switched to Digital Ocean where the card that Google declined was working perfectly. Anyway, I frantically setup a server on Digital Ocean and lost over 500 of the most recent chat messages since it was restored from an older backup. Finally, the site was back up.

Since I had closed registration and the launch day traffic started it’s journey towards the x axis, things became quiet in the chat room. Over the next few days, I implemented some basic anti-spam measures :

  • recaptcha on the signup and login forms

  • rate limiting : each user can send a maximum of 10 messages per minute

  • automated deactivation of accounts that use offensive language. Ironically, I used a Google API for this.

Once I implemented these changes, I reopened registration. On the 19th of January, a spammer noticed this and created an account with a username that had racist terms. He used his automated script to start spamming but ran into the rate limit of 10 messages per minute. Once the rate limit was reset after a minute, he continued spamming racial slurs before his account was automatically banned and I received an email with the subject line “User [****] banned”. The anti-spam measures worked! I logged in to the chat room to find that he had created another account with the username “bee_movie_ama”. Since his username had “ama”, I decided to ask him a few questions. We had an interesting conversation and it turns out that he was also the “bee_movie” user from launch day and he was also the user “arnold”. I asked him a few questions about 4chan, trolling and a few other topics. It was an interesting conversation, read it here. I also asked him about the spamming tools he uses and it turned to be a simple bash script that works as follows : manually create an account on the site and keep it open on the browser, then run xdotool to read from the beemovie script and enter it into the input form. That’s it, nothing fancy.

So what was the point of this blog post? It’s mostly an interesting story to me since this is the first time being targeted by spammers. If there’s something to be learned from this story, it’s that don’t build spammable websites! At the very least implement recaptcha and rate limiting for all websites with user generated content such as chat rooms and forums.

If you enjoyed this post, you might want to check out my chat room Hackers Chat where we discuss topics related to programming and technology. I’m exploring the idea of public chat rooms with this project, and have a lot of plans in the works! Join the chat room and let me know what you think.



     


© Plogging Dev - Powered by Hugo Theme by Kiss