This is a new one....

Introductions, comments, offers for catalogs - scans - documentation, and questions not answered by the FAQ (link at top of every page).
Post Reply
User avatar
Crusty
Virtuoso
Posts: 2324
Joined: Sun 15 Jan 2006 04:52 PM
Location: Absolute Stupidity, Missouri
Contact:

This is a new one....

Post: # 82215Post Crusty »

Got a "resource limit is reached" error trying to open the forums. Anyone else getting this?
User avatar
Jorg
Virtuoso
Posts: 3220
Joined: Sat 25 Mar 2006 09:25 PM
Location: Grand Absurdity, TX

Re: This is a new one....

Post: # 82216Post Jorg »

I've seen it a few times recently.
Quality service since 3:00 last Tuesday
Kmaxpe1500
Virtuoso
Posts: 285
Joined: Mon 19 Apr 2021 07:23 AM

Re: This is a new one....

Post: # 82217Post Kmaxpe1500 »

Yes, I think it started for me on Tuesday
User avatar
Crusty
Virtuoso
Posts: 2324
Joined: Sun 15 Jan 2006 04:52 PM
Location: Absolute Stupidity, Missouri
Contact:

Re: This is a new one....

Post: # 82218Post Crusty »

I checked our disk and bandwidth usage and we are within limits. If it happens again to you before it does me try to get a screenshot or copy the text. Post the text and if screenshot I can give you an email address to send the screenshot to.
User avatar
Barry
Virtuoso
Posts: 4039
Joined: Wed 29 Apr 2009 12:32 PM
Location: St. Catharines, Ontario Canada
Contact:

Re: This is a new one....

Post: # 82219Post Barry »

Yup!
Been getting it regularly for a few days, including just before I managed to login now.

Text message only, no server error number, just try again later. :evilrazz:
"A little song, a little dance. A little seltzer down your pants." -Chuckles the Clown
Guitars: https://legend.barryeames.com
Music/Pix/Videos: https://getback.barryeames.com
User avatar
Crusty
Virtuoso
Posts: 2324
Joined: Sun 15 Jan 2006 04:52 PM
Location: Absolute Stupidity, Missouri
Contact:

Re: This is a new one....

Post: # 82221Post Crusty »

Well, I tracked down the issue and of course Webline was zero help. They SHOULD have seen the problem long before I found it.

RAM and CPU were maxed out so I couldn't get in to take the models section offline to troubleshoot. I grabbed the raw server logs and wham, it was staring me in the face. YandexBot (Russian) was literally hammering the site! They were grabbing EVERYTHING and not being nice about it. Multiple requests from numerous IPs per second. The SQL server just couldn't keep up. Fortunately they do obey the robots.txt file and once I disallowed them the issue went away in seconds. I tried to tell Webline there is no way in h-e-double toothpics we could eat up that much resources. Instead they suggested I contact a web developer to look into the problem. What am I, chopped liver?

Moral of the story: Don't bother asking the experts, you probably know more than they do anyway.
User avatar
Barry
Virtuoso
Posts: 4039
Joined: Wed 29 Apr 2009 12:32 PM
Location: St. Catharines, Ontario Canada
Contact:

Re: This is a new one....

Post: # 82223Post Barry »

Great customer service eh? :roll:

Bloody robots.
Glad that you were able to solve the problem, and lucky that the robot was obedient, or then what?!
"A little song, a little dance. A little seltzer down your pants." -Chuckles the Clown
Guitars: https://legend.barryeames.com
Music/Pix/Videos: https://getback.barryeames.com
User avatar
Crusty
Virtuoso
Posts: 2324
Joined: Sun 15 Jan 2006 04:52 PM
Location: Absolute Stupidity, Missouri
Contact:

Re: This is a new one....

Post: # 82224Post Crusty »

He was probably smoking a Beedi at the time. Broken English.

Most major crawlers are fairly courteous. Google will grab a page or two then wait a while then another. Sometimes several days in between. Other literally scrape the site in a single go. Can you imagine how many pages they were grabbing per second to peg the CPU and RAM like that? Rapid fire.

Since the CPU and RAM were pegged I couldn't get into the stats to see who was hitting the site but the raw logs were accessible and that's how I figured it out. Pages of them sending request after request. Log downloads wasn't hindered. But, that's about all I could do. I had them kill all the PIDS connected to the site but that only stopped it for a little while.
Post Reply

Return to “Comments, Contribs, and such about the Site and Message Board”