#newznab

Good bye and take care - NZBs dot ORG #usenet #nzbsdotorg #nzbs #newznab https://www.instagram.com/p/BwYxBvXFZzY/?utm_source=ig_tumblr_share&igshid=18id0tbmjc7w8

That will update your regex's so update_releases.php will create some releases. If you're happy with Newznab I highly suggest donating and getting Newznab+. It comes with lots of improvements and you're directly supporting the people who work on it.

Over the last month a new development cycle has been underway. Moving all the “backend” process into a single daemon that operates as a service. The requirements were simple: develop a service that consists of a single daemon that is truly threaded, support modules loaded as libraries into the precompiled source, is easily deployable for a distributed environment, contains maintenance and essential monitoring utility, and works. Okay so the requirements weren’t that simple. But after a lot of brainstorming, trial and error, and a fifteen page road map it finally exists. The indexd daemon now supports split roles distributed across multiple hosts, executesore efficiently and overall is more stable. Although it is not operational on our production site yet, it is one of the first major milestones completed on the road to the new platform. Being able to compile it with a variety of additional modules or load them in as modification libraries on top of the original binary is extremely stellar and it is so much more stable. One of my favorite new features is internally labeled, “killbill” which is functionality that monitors the performance of the process and if various aspects fall below specific thresholds, it can email alerts, SMS, restart individual aspects of the service as well as reconfigure itself to make up for whatever is temporarily stealing its mojo. Additionally, if it finds itself hung — it reloads itself and carries on like nothing ever happened. Numerous of our currently proprietary APIs were directly included via mods/libs as well to provide even quicker access to essential data necessary to rename hashed releases, trash spam, and more. It includes a higher level of deep inspection into multi part rars. Queueing up suspected malicious, passworded releases and overly sized spam on to a cliche named data procurement server to download and fully inspect. We also have included a newly discovered technique for inspecting rars multiple levels deep to look for passwords all while downloading less than 30% of the data required to do it now. Tired of malware found within releases attempting to turn your machine into a BTC mining zombie? Good, because it wasnt making us much money anyway (ha, someone accuses us of that before). We now have ClamAV integrated and are continuong to look for additional measures we can take to protect our users. Overall it is a huge accomplishment and we are stoked. Another small piece that shows we are the best and that it does not take $15,000/month to operate an index. -addition/edit: Maybe it wasn't obvious, but it also is responsible for actually indexing and populating the database. We utilize a small subset of regular expressions to grab specific content from various groups and have incorporated a few other means to allow is to continue indexing without relying soley on expressions. We match from existing nzbs releases, a self aware release discovery scraper (no real name yet), manually added releases, and a variety of other mechanisms we do not wish to reveal.

As more and more Usenet indexers come online, you notice one absolute that all of them have in common: hashed releases are a pain in the ass. There have been a few modest attempts at renaming these hashed releases with only minor success. The backroom discussions usually involve a few people trying different techniques to rid their databases of this wretched plague. But, none have been extremely successful. So, I have taken it upon myself to begin developing an API as well as a regularly updated downloadable dump of my entire hashed releases database. The hashed releases table goes back about four years and is sourced from multiple locations. First, unlike the majority of indexers, I have the hardware, network and development resources available. The API is REST and the data resides in a SQLite3 database which will likely end up being migrated to a distributed MongoDB platform. The backend for the API is developed using PHP5 with Slim's light and elegant framework. I chose Skin because it is easy to deploy and had all the route features I needed. When a request is made with a given hash passed it will return HTTP 200 as well as the corresponding title for that release. The result is stored in Memcache for quite a while because it sill likely be requested multiple times in a short period. The database is hosted in memory with frequent commits to the filesystem for integrity's sake. I will write proper documentation as well as how to obtain an API key once I feel it is ready. The API currently only serves our index and I believe it is time to give back to the community. Additionally, the data will be provided to NZBVortex so it can be included as a dictionary of sorts for their users to be able to make sense of hashed releases.

It is of no surprise that the recent legal rulings within the copyright infringement world have cause a bit of flutter. Indexers are going offline and many more are bringing themselves online. It seems building an index is the new fad and everyone believes there is money to be made. I have a secret about that. There is no money, not if you are interested in developing a sustainable index. One that can hope to withstand the turbulent waters of the internet and it’s relentless policing forces. If you knew me personally, which for most of you is very unlikely — it would be known that I wish everyone great success in their ventures. But for most of the people behind indexing, I simply desire that they cease in their efforts. If Usenet as a source of content is to remain a feasible source and indexers the medium to assist users in finding that source then having too many is harmful. Even worse, having many more not knowing what theybare doing is detrimental to our overall cause. I know what I have built and the users who appreciate it do so because of the endless amounts of effort that goes into it. Many of them, I would suggest 10% of them have made serious attempts at organizing their own smaller and more private indexes. But the truth is, it takes a large amount of work and knowledge. Using Newznab is one thing. You install it, configure it, and you have an index. But securing it and keeping your content safe is another beast. Newznab has many factors that are inherently insecure. Those need to be patched. So these users have allowed their leased servers to lapse in payment in exchange for an affordable, but more expensive than others, index. A country club of sorts. It is definitely an elitists index with a community of knowledgable and loyal users. The type of users you would expect from a more mature site. I am proud of the community I have been building and they are proud to be members. Our niche has been carved out as an indexers index. An index that maintains a secure and stable environment. We are hidden behind many fronts. I have covered just about everything I can short of background checks on users. But if you do intend to move forward indexing Usenet. Pay close attention to what everyone else is not doing. Keep clear of SEO as it will only bring grief. Pay no mind to those placing Adwords ads on Google and make sure a review of your site appears nowhere online. Be synonymous with secret. You’re a society of individuals who wish not to be known. Each user whom relies on your expertise believes you are looking out for them. See to it that you are. Remember, it is easier to prevent than it is to repair.

During the last month I have been experimenting with a new process to backfill groups that are users wish to have on the site. The newest version is cruising along quite well and I have fixed the issue which caused the glitch earlier in the week. Most of the time our backfilling was halted was simply because I did not have time to check it out. However, an additional pass has been added to the backfill procedure which allows us to find far more releases. When compared to the method most are using with Newznab itself, we are finding more than 35% more releases. This means that I am having to store our unused binary parts longer than I would like so they are available for the second pass. In time this procedure will be optimized and developed to be more efficient snd effective. But for now, it means more content for very little additional energy or resources.

I have a pretty generous spirit and am a huge believer and practicer of pay-it-forward. I have some useful new security tips, a few new Newznab related tutorials and some general helpful information to share very soon.

Over the last month I have been using a new methodology and practice for fetching older content into the index. It utilizes more hardware, but is much more efficient. We have two or three medium sized groups to finish and then two much larger groups: - alt.binaries.cores - alt.binaries.boneless Despite having 90% of multimedia, teevee, x264 and hdtv content. I am going to restart those backfills for a second round soon. We also just completed a.b.erotica despite it appearing to have 1,544,230,100 binaries remaining. The release counts had not changed in four hours so I continued to monitor it and killed it. Additionally, once the content from these newly filled groups is imported we will have more content than any group of people could ever need. I will continue to update as the progress continues.