this post was submitted on 29 Apr 2025
443 points (97.4% liked)

Technology

69491 readers
3988 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
 

The one-liner:

dd if=/dev/zero bs=1G count=10 | gzip -c > 10GB.gz

This is brilliant.

you are viewing a single comment's thread
view the rest of the comments
[–] palordrolap@fedia.io 92 points 1 day ago (3 children)

The article writer kind of complains that they're having to serve a 10MB file, which is the result of the gzip compression. If that's a problem, they could switch to bzip2. It's available pretty much everywhere that gzip is available and it packs the 10GB down to 7506 bytes.

That's not a typo. bzip2 is way better with highly redundant data.

[–] just_another_person@lemmy.world 78 points 1 day ago* (last edited 20 hours ago) (1 children)

I believe he's returning a gzip HTTP response stream, not just a file payload that the requester then downloads and decompresses.

Bzip isn't used in HTTP compression.

[–] sugar_in_your_tea@sh.itjust.works 20 points 22 hours ago* (last edited 22 hours ago)

Brotli is an option, and it's comparable to Bzip. Brotli works in most browsers, so hopefully these bots would support it.

I just tested it, and a 10G file full of zeroes is only 8.3K compressed. That's pretty good, though a little bigger than BZip.

[–] sugar_in_your_tea@sh.itjust.works 20 points 22 hours ago (1 children)

Brotli gets it to 8.3K, and is supported in most browsers, so there's a chance scrapers also support it.

[–] Aceticon@lemmy.dbzer0.com 3 points 15 hours ago (1 children)

Gzip encoding has been part of the HTTP protocol for a long time and every server-side HTTP library out there supports it, and phishing/scrapper bots will be done with server-side libraries, not using browser engines.

~~Further, judging by the guy's example in his article he's not using gzip with maximum compression when generating the zip bomb files: he needs to add -9 to the gzip command line to get the best compression (but it will be slower).~~ (I tested this and it made no difference at all).

[–] sugar_in_your_tea@sh.itjust.works 3 points 12 hours ago* (last edited 12 hours ago) (1 children)

You can make multiple files with different encodings and select based on the Accept-Encoding header.

[–] Aceticon@lemmy.dbzer0.com 1 points 12 hours ago

Yeah, good point.

I forgot about that.

[–] some_guy@lemmy.sdf.org 4 points 1 day ago

TIL why I'm gonna start learning more about bzip2. Thanks!