Site stability fixed (I think)

Important game and board announcements and website or forum related topics

Moderators: The Heralds, The Loremasters

Post Reply
Message
Author
Daeron
Scotty? Laforge? Kaylee? Engy? Evil Wrench?
Posts: 107
Joined: Thu May 04, 2006 10:47 pm

Site stability fixed (I think)

#1 Post by Daeron »

Greetings everyone,

For the past weeks, both Ulthuan and Druchii sites have been suffering from downtime. I want to apologize for the inconvenience.
The source of the problem was that AI bots were scraping our forums, to the point that they completely shut down the normal working of the sites. I've blocked their access, and it seems the site is acting normally once more.

Unfortunately, our forum is longer being used to train AI LLM's. As a result, AI bots might become less versed in Warhammer and may start talking like normal people again.

If there are any other issues, please feel free to point them out and I'll have a look at it.
Cheers,
Daeron

--- For those interested in more technical details

Our site was struggling to communicate with the outside world (sftp and ssh access were still fine). This lead me to believe there was a congestion for HTTP. The Host support indicated the site was functioning as expected, and that there may have been "too much" access. The site had over 700K visits yesterday. It took me a while to figure it out, but the access logs showed a lot of bot activity. Some googling showed it's AI LLM training related.
Over 600K (88%) were from a single bot source.

I tried a different strategy for Ulthuan and Druchii, to see which one would be more effective. For Druchii I tried the nice way, by blocking the AI bot through the robots.txt file. Bots usually check this to see what they are and are not permitted to do. It seems this one, Claude, wasn't interested in playing nice. For Ulthuan I tried a Firewall rule to block Claude. Unfortunately, this isn't supported by our Host without paying a premium. It is possible to secure this at the DNS level though. So I set up the CloudFlare account to provide our DNS services, and set up the blocking rule there.
The traffic dropped almost immediately, and the firewall rule was triggered roughly 2000 times in just a few minutes.
Since the firewall is up, forum behaviour is back to normal.

The downside is that we're now dependent on an external party for the DNS service, but it gives us free (but limited) Web Application Firewall. This shouldn't be an issue at technical level. I just prefer to keeps things centered, but this was getting out of hand.
Oops
SpellArcher
Green Istari
Posts: 13866
Joined: Sat Sep 13, 2008 11:26 am
Location: Otherworld

Re: Site stability fixed (I think)

#2 Post by SpellArcher »

Thanks Daeron.

The site's a little slow for me but it is working. Appreciate your hard work here.
User avatar
Anduil of Elithis
Posts: 247
Joined: Mon May 21, 2007 8:25 am
Location: Hamburg

Re: Site stability fixed (I think)

#3 Post by Anduil of Elithis »

Thanks for fixing this. I was wondering what was causing these issues. Good that we are back to normal.
User avatar
TyrrenAzureblade
Posts: 409
Joined: Thu Aug 06, 2020 11:55 pm
Location: Seattle, WA
Contact:

Re: Site stability fixed (I think)

#4 Post by TyrrenAzureblade »

Sweet Vaul, thank you for that. I was on vacation and couldn’t catch up on anything going on here.
Post Reply