Get Billions of Text and Binary Files, Over 110,000 Newsgroups. Save 58% on NewsHosting now!
What Is Leafnode?
Leafnode is an offline proxy for USENET servers. It downloads news posts from a remote news server and serves it as a local service in your system. It can also take multiple USENET servers as its upstream sources and merge them together to serve as a single source. This is useful if you want to subscribe to multiple newsgroups being served by different providers. Further, Leafnode allows the user to have offline copies of the followed newsgroups. This is important when an Internet connection is not always available or if you are looking to archive the newsgroups you subscribe to. Lastly, Leafnode is simple, easy to set up and works with all modern USENET readers.
How Does a USENET Proxy Work?
As we mentioned above, Leafnode is a proxy USENET server, so it essentially acts as the news server for your news readers. However, unlike a traditional USENET server, Leafnode does not obtain the full copy of all the hierarchies from other servers. Leafnode only gets the posts that you tell it to get under the conditions you have set. Leafnode interacts with two programs: fetchnews and texpire.
Fetchnews is the program that tells Leafnode which newsgroups to subscribe to and how many it should get. It also uploads your news posts and replies.Texpire is the maintenance program that removes old posts from newsgroups. It also repairs duplicate or broken messages and updates the overall data for the news server.
These two programs allow Leafnode to run mostly unattended. Once you are done setting it up, Leafnode will behave as if it is a remote USENET server. This also means you can set up Leafnode in a small network and allow multiple users to connect to it to read news posts. You can, therefore, reduce the amount of bandwidth your network is using to access USENET.
Installing Leafnode
Leafnode is designed to be as simple as possible. This applies to both its installation and maintenance. With that, installing Leafnode is just a matter of getting it from your distribution’s repository. For example, in Debian and Ubuntu 21.10, you can get Leafnode from apt:
Setting Up Leafnode
During installation, Leafnode will ask you about a few details of the remote server you want to connect to. In the first screen, it will ask you about the address of the remote server. For example, if you want to connect to AIOE, you need to put nntp.aioe.org here. In my case, I am connecting through Eternal September, so I am adding the address here.
Setting Up Automated USENET Downloads
From there, the installer will ask you about your preferred method of fetching the news posts. You have three options here: PPP, Permanent and None.
PPP will automatically download new posts whenever you connect to the Internet.Permanent will automatically download news posts every hour.None will not enable automatic news fetching. You will have to run the fetchnews command to synchronize with the remote server.
In my case, I do not want Leafnode to automatically connect to the news server without my knowledge so I am picking None.
Enabling Access Controls
The Leafnode installer will ask if you want to enable Access Controls for your server. This is relevant if you are using Leafnode as a local news server for multiple users. It allows you to use the /etc/hosts file to control which clients can access the server and which cannot. For example, if you are using Leafnode in an intranet where you expect each client to access the news server, there is no need to enable “Access Controls.” Despite that, it is generally good practice to block all incoming connections and only enable the ones you want. Further, if you are only using Leafnode as a local proxy for your own machine, then you have to enable this option. In my case, I will only be using this instance of Leafnode in this machine, so I will enable Access Controls.
Initial Fetch for USENET Groups
After that, the Leafnode installer will ask whether you want to connect to the remote USENET server now. This will allow Leafnode to get the list of the available newsgroups in the server. However, if you are using a service that requires authentication, such as with Eternal September, you should not fetch any newsgroups using this option. This is because we have not provided our account information to Leafnode yet, and updating right now will not give us the complete list of newsgroups. In my case, because I am connecting through Eternal September, I am picking No.
Configuring Leafnode
From there, you can further fine tune certain aspects of Leafnode’s behavior through its configuration file. For example, there are settings that will allow you to, among other things, use a USENET account to fetch the news. You can also set certain parameters that will tell Leafnode which posts it can download and how long to store it. The location of the configuration file varies depending on the Linux distribution you are using. In Debian and Ubuntu it is located at “/etc/news/leafnode/config.”
Setting the Default Username and Password
The configuration file for Leafnode is extensive and well-documented. Each option has a brief description of what it does, and some even have examples that illustrate how to set it properly. To use a USENET account in Leafnode, you need to uncomment the username and password variables. When finished, set your information in them. In my case, I have added the account information for my Eternal September account.
How to Create Post Filters
One key feature of Leafnode is that you can manipulate how posts are treated while being downloaded by either filtering the posts or setting certain newsgroups to not be pruned after a number of days. Filtering posts can come in two ways:
Based on the general properties of a given post. For example, you can filter by the amount of lines as well as the size of a post.Based on specific terms found in the post headers. This is done by creating a list of regular expressions in a filterfile.
To filter based on general properties, enable them directly from the configuration file. For example, you can set the minlines variable to 3, which would tell Leafnode to not get any posts that are less than three lines in length. On the other hand, to filter based on specific words, you need to enable the filterfile option in Leafnode. Do this by uncommenting the filterfile variable and providing a path to your filter file. In my case, I am using the default path for the filter file. The basic syntax of the filterfile looks something like this: Each line in the filterfile should point to a certain header property of a post. For example, if you want to filter all posts from Google Mail, type something like this:
How to Archive USENET Groups
By default, Leafnode prunes all of the groups for posts that are older than 20 days. This allows Leafnode to keep its file size down and is useful when you are subscribed to groups that see 100 to 200 posts a day. However, there may be instances where you want to archive all of the posts in a given newsgroup. Do this by enabling the groupexpire variable. For example, if I want to archive the comp.unix.misc newsgroup, I will add the following line to my configuration file: Furthermore, this variable can also be used to change the default pruning age for specific newsgroups. For example, if I want to prune a high volume group like misc.internet.news.discuss every five days, I can type this line to my configuration file:
How to Set Up Multiple USENET Servers
As we discussed above, Leafnode also allows its users to fetch posts from multiple remote servers. This is useful if you use USENET readers that can only connect to one server at a time. Further, you can also set a number of options that change the behavior of the additional remote servers. You can set the nopost variable to 1 to make it so that a remote server would only fetch posts and not send posts through it. On the other hand, you can set the variable noread to do the opposite. For example, if I want to set Aioe as an additional “read-only” server for my Leafnode instance, I can type the following in my configuration file:
Creating a cronjob to Update Leafnode
Aside from the automatic download script, Leafnode can also be updated through a cronjob. This allows us to have more flexibility when we want to update our newsgroups. For example, we can make it so that Leafnode will update every 12 hours and when we start our computer up. For that to happen, we need to edit the crontab of the news user. The news user is the system account that directly manages the “/var/spool/news” folder as well as the running of the Leafnode server. To edit the news user’s crontab, we can run the following command: We can now insert the command we want the news user to execute regularly. An example crontab for Leafnode looks something like this: This crontab will execute the Leafnode maintenance program, texpire, at 3 am every Sunday but will also run the fetchnews program once every 12 hours. Congratulations! You now have a basic knowledge of using Leafnode as a USENET server proxy. With that, if you are looking for some USENET readers for Leafnode, we recommend these five options. Get Billions of Text and Binary Files, Over 110,000 Newsgroups. Save 58% on NewsHosting now!
1. I’ve downloaded Leafnode in Debian but made a mistake with my initial installation script. How can I go back to it?
Use the dpkg-reconfigure program to reinitialize the installation script for Leafnode. This program, however, is not included in the default PATH variable. To run it, you can type the following command:
2. I want to run Leafnode in my LAN. How can I do that?
You can do that by editing both your “/etc/hosts.deny” and “/etc/hosts.allow” files. This allows you to filter all the incoming connections to your Leafnode server. An example of a hosts.deny file looks something like this: On the other hand, the hosts.allow file that allows for local connections that uses 192.168.254.* looks something like this: You will need to change the allowstrangers variable to allowSTRANGERS and set its value to 42.
3. I want to download many posts when I first connect but don’t want them filling up my disk space. Is there a way I can do this?
Yes! To do this, you only need to be aware of two variables: initialfetch and delaybody. If you want to download all of the posts from a particular newsgroup, comment the initialfetch variable to tell Leafnode that you want to get all available posts from the remote server in this newsgroup. However, downloading whole posts can take up a lot of space. You can reduce this by just downloading the headers of the posts and the bodies of the posts you are interested in. For that, you need to set the delaybody variable to 1.
