Stuff I'm Up To

Technical Ramblings

Updating Newznab — December 23, 2016

Updating Newznab

Newznab is a Usenet indexing server. It’s a very powerful spider that grabs details of all kinds of Usenet posts in the groups you’re interested in. It then indexes and stores them as NZB files so you can download complete releases.

I’ve found that running it helps me find books as not many out there seem to interested in indexing the book groups. It’s probably not of much interest to you unless you’re also looking to index content that isn’t common place. Most indexers out there cover Music, TV and Movies – so you’ll probably find little use for it.

As I don’t actually run it that often, because it’s not everyday I’m looking for a book. Updating it periodically is something I often forget how to do properly.

$ cd /var/www/newznab
$ svn update .
$ cd misc/update_scripts
$ php ./update_database_version.php
Usenet — September 22, 2016


So what’s Usenet?

Usenet has been around since God was a lad. It’s been part of the internet for so very long it pre-dates all the graphical stuff we see today.

Put simply it’s a text based system of sharing news messages , hence it’s correct protocol name being Network News Transfer Protocol (nntp), and operates in a similar way to a forum. Text posts would be made into news groups containing newsworthy information and could be responded to by people all around the internet. The news servers replicate and spread this news between each other so any piece of news may exist on many servers throughout the world.

Back when everything started becoming more than just text the cleverest propeller heads started to figure out you could convert binary files into a text format and back again. So binary files could be attached to email and sent as text and converted back from text at the other end.

Well the same was true for Usenet messages. So it didn’t take long before the text based system became host to binary files converted to text. This meant the underlying Usenet system itself needed no changes. It would still replicate these converted text messages across the globe. But now they could be converted back to binary files and the global sharing of anything from pictures, movies and music became an ingrained part of Usenet.

Continue reading