#zstd

Zstandard is pretty amazing. It's not going to win any awards for compression ratios (lzip takes that prize ime) but it's FAST.

Really fast.

"Compressing a tar file of Web content from 5Gb to 3.6Gb in 6.9 sec" fast.

Fedora vai mudar forma de compactar mídia de instalação

Caiu por terra a proposta de compactar melhor a mídia de instalação do Fedora 32 via SquashFS. Isso já vinha sendo discutido há algum tempo e seria feito sem o chamado nested EXT4 file-system setup para suas imagens live, além de também aumentar a compactação XZ. No entanto, essa proposta foi rejeitada na reunião de engenharia.A equipe alega que outro caminho de compressão mais ideal poderia ser…

Arch muda de XZ para Zstd: pacote cresce menos de 1% e descompressão fica 1300% mais rápida

O Arch Linux vem trabalhando nos últimos meses na transição para pacotes compactados com Zstd, no lugar da compactação XZ, para uma instalação mais rápida dos pacotes. No final de dezembro, o esquema de compactação de pacotes mudou e os resultados são impressionantes. Portanto, o Arch muda a compactação de XZ para Zstd, pacote cresce menos de 1% e descompressão fica 1300% mais rápida.

Ataxa de…

All the political stuff has been getting me down, so here’s a tech post. 

I love non-cryptographic hashes. It’s a hard problem with tons of research and objective results. Whereas cryptographic hashes have to worry about stuff like collision attacks and that’s no fun. In the crypto world, speed can be your enemy (e.g. password hashing). But in non-crypto it’s all about speed and how well you cover the keyspace. 

Turns out that Yann Collet, author of xxhash and lz4 compression came out with a new thing, zstd (code) last year when I wasn’t looking. This may be old hat to some of you all out there, but I think it’s pretty cool. It’s in the same space as zlib (i.e. gzip), which is to say medium-fast, medium-space, vaguely symmetric cost for compress vs. decompress. It does all kinds of neat tricks to be a LOT faster than zlib at similar compression ratios. The linked article goes into more detail. Spoiler alert: it’s pretty rad if you are a massive geek.

It’s also got some cool dictionary building stuff, so under some loads you can get massive compression improvements, similar to what Facebook published before with their preview image optimization.