A Cloudflare spokesperson told Ars that the cloud services provider saw “a spike in unusual traffic to one of Cloudflare’s services,” which “caused some traffic passing through Cloudflare’s network to experience errors.”
“We do not yet know the cause of the spike in unusual traffic,” the spokesperson said. “We are all hands on deck to make sure all traffic is served without errors. After that, we will turn our attention to investigating the cause of the unusual spike in traffic.”
translation: we don’t fucking know
“You see, half the internet went down because people used our services a little too much.”
Ok wtf???
How does cloudflare not have DOS detection?
That was based on anecdotal evidence, at the time. The real cause was Cloudflare shooting themselves in foot again because of a bad config file.
The bad config file is somewhere in the middle of the chain of causality.
They changed database permissions, revealing a dormant bug in a database query, leading to config files being generated badly with duplicate lines, making them too large for intake by the bot detection service, which didn’t have good input validation and made the process panic instead, ruining the service.
Widespread Cloudflare outage blamed on mysterious traffic spike
Just like my 14-year-old self who “mysteriously” bricked his parents’ PC by totally SFW Web usage.
- Double clicks h4rdc0r3p0rn.exe right from limewire
Double clicks h4rdc0r3p0rn.exe right from limewire
Quickly followed by existential dread after realizing what I’ve done.
All the people trying to cash out their Dr. Pepper points?
Did someone google Google again?
Aren’t spikes in unusual traffic the exact thing Cloudflare is supposed to protect you from?
Traffic spikes, on the Internet? One in a million chance! Now tow cloudflare outside the environment and call it a day.
They protected the endpoints. They just weren’t able to route traffic to them. Id bet it takes a MUCH larger ddos to bring cloudflare to its knees vs your average website.
From a Cloudflare customer’s point of view, I don’t care if my site is down from a DDOS or a Cloudflare outage, but the latter seems to happen more often.
As it stands? Cloudflare is still incredibly effective at protecting customers from those DDOS attacks. Which, depending on your hosting solution, can mean very noticeable monetary savings because YOUR hardware/connection didn’t spike. And, regardless, can mean noticeable monetary savings as your engineers didn’t need to recover a crashed system because your setup was just sitting there idle.
That said: If you truly need high availability? You need to do what downdetector did and have alternatives ready in the event that Cloudflare falls over. Same as with your ISP… which should be ISPs plural.
From another cloudflare customer, if our sites still work internally it’s marginally better than them being broken both inside and outside the org as they would be if they were ddosed directly. I guess it depends on what kind of services you’re running.
Yeah, but just one “unusual spike in traffic” - so it seems. /s
Task failed successfully.
Fission Mailed
ostensibly sure. But it’s like car insurance. People pay them no matter what so why bother doing what they promised?
Amazon is now saving Americans from the crippling debt most of them seem to get into to drive a shiny box… I wasn’t expecting that.
Buying a car is pretty capitalist.
I’m not envious and I like the thought of some places on the planet having it normal for one family to have few cars, but - in certain places where capitalism has already been disrupted a few times in history, like Moscow, Russia where I, eh, reside, am headquartered, dwell, roam, … - and those places are richer than much of the world, - people would think the guy from the screenshot doesn’t have many reasons to try disrupting it.
Wooooosssshhhhh
My bad, I started downloading The Lord of the Rings movies - Extended Edition. Sorry!
How many times have I told you not to download movies or games in the middle of the day? You’ll tie up the phone lines.
Ok mum I stopped torrenting. You can use the phone again.
You can tell we’re all old as fuck Millennials, because nobody else would make this joke. Lol.
I can’t even explain dial up modems to my son because I’d have to start by explaining what phone lines are.
It wasn’t about the lines. It was always about the switches. And while they are no longer actual hardware, but rather software, those are still the what makes phones work.
Indeed.
“…you must first invent the universe”.
I had a Gen Z person ask me how I got a “3D printer save button” when I had a floppy disk for some reason.
I think that’s a big part of why I like lemmy.
There are plenty of tech-savvy critical thinkers in the younger generations, but the naïveté, tech illiteracy, and lack of critical thinking ability of the average internet commenter / poster is appalling.
I’ve seen it just get worse and worse.
The internalized self censorship, the laissez faire attitude towards digital privacy, just pure fucking idiocy.
Wake me up when September ends.
Sorry friend, it is Eternal.
I am 26 and this is my childhood too.
No longer relevant pro tip - an extra pair was typically left at the incoming service, which was used for testing. It worked as a second line and didn’t interupt the main pair, allowing for a functionally free second phone line (that didnt have incoming service).
Perfect for modem use!
I had fully forgotten the phrase “you’ll tie up the phone line!” And I just had a nam style flashback of sneaking internet time during the day when my mom was at work, and praying that no one tried to call
Ten movies streaming across that, that Internet, and what happens to your own personal Internet? I just the other day got… an Internet was sent by my staff at 10 o’clock in the morning on Friday. I got it yesterday. Why? Because it got tangled up with all these things going on the Internet commercially. They want to deliver vast amounts of information over the Internet. And again, the Internet is not something that you just dump something on. It’s not a big truck. It’s a series of tubes. And if you don’t understand, those tubes can be filled and if they are filled, when you put your message in, it gets in line and it’s going to be delayed by anyone that puts into that tube enormous amounts of material, enormous amounts of material.
Lies, the internet isn’t tubes, it’s a small box
Hey me too!
Which AI scraper went rogue this time?
It was actually the system Cloudflare uses to catch and block bots that went haywire.
They had a fake database you could query that would pull content from a bunch of different shard databases. They updated the config so that systems querying it could see the shards in addition to the main dummy DB. The tool that pulled data out of it assumed that it could only see the dummy, however, so it just asked for everything when it pulled a report to pass to the filtering system.
The filtering system assumed the report it received would be properly formed and crashed if it got one that was malformed.
Gemini 3 was released today 👀
Mabe the new bezos Prometheus?
Did I miss something? Is everyone downloading the Epstein files today?
I hope more websites will move away from cloudflare. I could not access 90% of the web anymore. This is insane if just 1 company goes down, the whole internet is dead. The internet is broken!
and it’s fucking annoying to check the box to “prove you’re a human” when trying to access almost any site. some days it will make me do it three times before letting me through
He still doesn’t know, guys 😅
🤖
I understand the need for anti-bot or DDoS protection, but there are better and free options today. Like Anubis. So please, in the love of The Internet, move away from cloudflare. Ideally yesterday already.
Edit: or run your own decent firewall with geo blocks. FireHOL block lists. Intrusion detection.
Setting up fail2ban. . Etc. Etc.
Anubis isn’t even comparable to cloudflare. The reason cloudflare is so effective is that they can oversee which IPs are spamming or being abusive to certain websites, and can throw up protections quickly. There are a number of negative implications that come with this, but it’s quite good at its primary job.
Anubis is just a prompt that wastes CPU cycles and tries to make it more expensive for AI crawlers to do so (since they care a lot about compute costs, of course). There is no bot protection or anything happening. The “making sure you’re not a bot” is quite misleading imo
Anubis is to protect against scraping from LLMs, it has nothing to do with DDoS protection. Not only that, but the Anubis Github repo recommends most people to use Cloudflare instead, since Anubis is the “nuclear” option.
Well then we are all fked. I recommend then using a good internet connection to host your stuff behind it. I also recommend a good firewall.
A firewall that can block on geo location. Block on ASN level. And intrusion detection. And also use block lists like FireHOL level 1 to 4.
Of course configure fail2ban etc.
Again we really need to come up with alternatives now… I’m sick of how the current internet develops.
Anubis never worked for me on mobile. I’m afraid of mass adoption if that won’t be fixed.
It isn’t just annoying, it often breaks for people on less-popular browsers. Plus, it requires you to run Cloudflare’s Javascript. You think this outage was bad—what do you think would happen if someone slipped them a bit of malware?
next time you’re at bat against one of these, you may try moving less diligently / efficiently to the checkbox. overall, a slowed and less exact approach. I’ve not tested this enough to REALLY say it makes a difference, but in cases where I continually fail, going slower does seem to be the time I finally get through.
I find the same for the picture puzzles where you select images that match or apply to the posted context or whatever else the mission may be.
It’s almost like the internet is a public good that everyone should have access to
Internet is so fucked up and manipulated that people would be better without it.
It’s very normal for countries to DDOS each other to test their limits, and if it could be incorporated in an attack.
And yes, of course the US is also testing this against other nations.
It could also just be a malfunction or someone acting independently. Who knows.
Every DDoS that hits Cloudflare is bigger than the previous one. There’s bound to be one that they couldn’t instantly mitigate.
Did Akamai, their competitor, ever blinked? (I don’t remember).
Even if they did, the importance of decentralising the internet away from a few big providers is no joke.
Yes of course I totally agree.
Capitalists: “All lines must go up!” … (Traffic line goes up a little bit) Capitalists: “Not like that!”
Soviets?
deleted by creator
