Did not see this coming when I built my 40TB NAS
Get to acquiring Seagate external HDDs and shucking them for your own 3.5" drive bays before the data centers get them
The 20TB drives I was looking at in July are up 40% :/
Sadly my wallet is on time out
Wasn’t all of this shit already available as torrents?
No.
Not everything got torrented after music streamers came into prominence. (Though chances are pretty good you could rip an MP3 off Youtube for whatever you’re looking for.)
It would be awesome if we had an app that allowed to stream directly from such torrents, and had a user-made recommendation system to replace the discovery algorithm :D
Stremio + Torrentio does this for TV but I haven’t found an equivalent for music. Hoping to be proven wrong 🤞
That’s nothing compared to my old Napster collection
Datahoarders are going to go WILD over this
data hoarders already have everything in here and far more, and the web release versions are a lower priority. thinking of red.sh here
I don’t think many data hoarders are sitting on the AI generated stuff 😁
Let’s put it all on a Funkwhale server.
Sure, you set it up.
That’s disgusting. Where would you find such torrents?
The album art torrent is a goldmine. Such a pain in the ass sometimes to find high quality album covers.
Have they actually been indexed?
People are saying it’s 300TB but this link is only 200GB why?
The 200GB is the metadata sqlite database only
God damn! That’s essentially just text, right? Or would it also include album cover art?
Basically, the id3 tags for the music files. However, Spotify uses several more nonstandard tags in their database, some of them are great to make playlists.
Not released yet
cue Padme
‘And avoid it?’ — ‘To avoid it, right?!’
Anna’s the GOAT
As far as I’ve read, the database is largely low bitrate files, and some AI. The value here is metadata and preservation of “rare” music.
Nope, I would not call 160kbps Vorbis low bitrate, it’s roughly quality of 192kbps MP3. Only the ”popularity=0” stuff (so stuff with so few listens that Spotify does not keep record of) were re-encoded to 75kbps Opus, which as a modern codec is much better than it sounds like but of course re-encode is not great for already lossless stuff.
For purists there are those Tidal downloader sites available everywhere for free lossless music, even 24-bit hires FLAC.
Opus is what I’m encoding my working library to. I like ripping to flac (and archiving them as such), but the advantages to smaller file sizes for the working library are worth it for me. So far, I’m really liking the format.
I keep the archive on spinning hard drives, but the opus library on ssd (which makes browsing much quicker, and no unnecessary spinning up the hard drives.)
It’s not lossless but current ogg vorbis at 160kbps is absolutely transparent for the vast majority of people. That’s actually what I chose to keep my own collection, I mean, outside of the lossless albums that I absolutely want to flawlessly preserve.
All tracks within the top 99.6% of listens are supposed to be high quality
not sure why you want that much music most of it garbage. i would like some of the podcasts that people dont post anywhere else though. all hail the data hoaders.
Is this new? Aren’t most tracks already available in torrents?
Not mine, because I’m not famous enough for people to pirate my music lol. It would be flattering for me to be included in this batch of scraped music.
I’d steal your music
If your Spotify popularity is not 0, you probably are in the scraped archive.
Yep, most of tracks were already available on “various” sources, but this time they directly scraped the whole Spotify database.
It’s really nice from them to backup Spotify database on a distributed system, and for free ! This ensure Spotify business won’t be endanger in case of critical hardware failure.
So nice of them to help with Spotify’s off-site backup.
It’s new insofar as this is one big scrape. About 300TB iirc.
300tb is a lot, but its kind of crazy to think this entire company only needs 300tb storage arrays to function. I wonder how they handle things internally. I would imagine at least 1 backup server ready to go in HA. I wonder if they have multiple regions across the country that also serves up the same setup.
Likely cloned Netflix’s “netflix in a box” design, where they drop a large 200TB+ NAS in thousands of different CDN datecenters with their most popular content cached so that total traffic is minimal across the internet at large.
Spotify mainly being music with very little video likely makes this even easier.
There are 245 TB ssd drives now. You can almost fit that in a single drive.
They need other 300TB to store all the ads.
“Are you an incel with few friends, no job, and a deep seated hate for melanin? COME JOIN ICE!”
Afaik 300 TB is just the most popular music and around a third of all tracks. The blog post on anna’s is quite entertaining tho.
IIRC there’s still like 700TB of low popularity music missing, but it is only something like 0.4% of listens.
And they need a more storage overall because they have to set up datecenters around the world - doesn’t make sense to stream tens of millions of connections across the ocean. But that also gives all the backups one would need for “free”.deleted by creator
Oh I know, I work in the industry as well. Our company backups alone for workstations and servers is just under 1 petabyte. This is then replicated to an offsite location which is also out disaster recovery location, and also stored in long term storage in Azure. This is just backups, sooo much money for backups haha. Thats why I am shocked that this entire company can run off of 300tb which is a lot, but nothing when you think of it being the entire business model for them.
I think the craziest thing ive seen is we have these instruments that do genome testing and sequencing and they would create like 10tb worth of data per month. Every month they got there own 10tb drive handed to them to backup their stuff on there own on top of the ones we did for them.
I wish I could think anything positive about this, but I can’t imagine anyone who actually cares about music needs or wants this. Instead it’ll almost certainly be used as an illegal and unethical dataset to further train bullshit AI to make slop songs. As easy as it is for people to claim “preservation”, I do have to question the motives of stuff like this…
Fuck AI. Support your favorite human artists.
Yeah, the people who are racing to download it all want to use it for profit. AI companies, companies that run databases, etc.
It’s a good thing if you are smart enough to understand that AI isn’t going away. Universal bought udio, the “legal” variant of the dataset will be used to train models, only they will be closed source, censored and come with a ToS that gives all the rights from the generated music to the record companies from the get go.
At least this gives open source a chance.
I mean, it seems like the perfect avenue to replicate Stremio / Kodi but for music
Same it seems useless to me. The real value is knowing how songs relate to each other in terms of being played before/after other songs, and that’s only available via internal datasets that they could never scrape anyway.
Unless…?
I guess it’s easier packaged in a torrent rather than individual downloads, but I do question why anyone would need this if all they were doing was training AI, as everything is already available on YouTube for free. You don’t need to hack a company to get the audio. Now if you’re a human trying to actually listen to the songs, obviously the Spotify torrent will sound much better, assuming the hack captured the higher quality audio streams from Spotify. All the youtube downloaders sound like crap because its like 128kbps m4a (at least on newpipe, though you can do 160kbps opus if you want)
Removed by mod
Everybody starts somewhere. Few come out the gate being Depeche Mode. That doesn’t mean it’s not worth the struggle to get better.
Exactly, bars just got higher so in a few years or so human artists will generally be even better, and they too will be chased by AI so the chain is created.
I find that unlikely when considering the current trajectory of AI.
You smoked some cracked when you found that out eh?
