Spread Logging - Jigdo Members

Installing Spread

First thing needed is to install spread itself. Fedora Unity has packaged spread and will be working to provide the bits via the repos. For now, we will provide links here.

What is Spread?

"Spread is an open source toolkit that provides a high performance messaging service that is resilient to faults across local and wide area networks. Spread functions as a unified message bus for distributed applications, and provides highly tuned application-level multicast, group communication, and point to point support. Spread services range from reliable messaging to fully ordered messages with delivery guarantees.

Spread can be used in many distributed applications that require high reliability, high performance, and robust communication among various subsets of members. The toolkit is designed to encapsulate the challenging aspects of asynchronous networks and enable the construction of reliable and scalable distributed applications.

Spread consists of a library that user applications are linked with, a binary daemon which runs on each computer that is part of the processor group, and various utility and demonstration programs.

Some of the services and benefits provided by Spread:

• Reliable and scalable messaging and group communication.
• A very powerful but simple API simplifies the construction of distributed architectures.
• Easy to use, deploy and maintain.

• Highly scalable from one local area network to complex wide area networks. Supports thousands of groups with different sets of members.

• Enables message reliability in the presence of machine failures, process crashes and recoveries, and network partitions and merges.
• Provides a range of reliability, ordering and stability guarantees for messages.
• Emphasis on robustness and high performance.
• Completely distributed algorithms with no central point of failure. "
  

From/Read More: http://spread.org/

Where are the Packages?

Right now, they are on Fedora People.

To be able to send data to our spread node, you need to install the spread package and the perl-Spread package from: http://jsteffan.fedorapeople.org/spread/

Also, make sure to disable the spread daemon as you wont need to have it running:

chkconfig spread off

To be able to read the log data you can either use spuser to view the bits, or use spreadlogd to write the logs out to disk. Fedora Unity will be running a spreadlogd to record node access logs. The results of these loggings will be public so there is not much need to log the bits to your local disk.

Setting Up Apache to Log to Spread

mod_log_spread is not working very well with Apache 2.x and also takes over all logging. This is not desirable for our use case so we have provided a perl based logger that you pipe your access logs to. It is the most lightweight and non-intrusive way to setup spread logging for a single vhost at this time.

Adding the Needed Directives to your VirtualHost:

Add the following in replacement of existing CustomLog directive(s):

CustomLog "|/var/www/client.pl -g jigdo" combined

Optionally, you might need to set -h to define the name of your node. Most systems will return a correct host name from the hostname command, but yours might not.

Gaining Access to Send Data to the Spread Node

To be able to send data to the node, you will need to contact us so we can allow your connection through our firewall. As a note, spread transmissions are non-encrypted and there is no user authentication. Any data you send via your node can and will be read by anyone. Restrictions via iptables is just a measure to ensure we only have legitimate reports of access logs from known admins.

Where is the Client?

The client for Fedora Unity Jigdo Members will be maintained in the rsync source. client.pl will end up in $basedir/unity/client.pl

Please note the following permissions must also be set:

chmod +x /var/www/client.pl
chcon -t httpd_sys_script_exec_t /var/www/client.pl