Introducing Cfruby, Cfyaml and libcfruby
Powerful next generation tools for managing large scale heterogeneous computing environments
1.1 Overview
The Cfruby project is an open source software initiative with the dual purpose of (1) providing a solid library for system administration and (2) tools for emulating and superseding the workings of GNU Cfengine.
Large scale networked computing environments need appropriate tools for managing system and computational services. Cfruby is a tool-set that allows full automation, transactions, granular locking, versioned backups, extensive and configurable logging, a state machine with Cfengine style classes and scripting.
The project has two types of users. The first type is the system administrator who wants to replace Cfengine with something more flexible and advanced. The second type is the Ruby developer, who wants to use powerful system administration features from his software. Since both worlds are served it means a broader user base which adds to the robustness and reliability of the libraries.
Here some concepts are introduced so you can get an appreciation for what can be done.
1.2 Cfenjin, a scripting language and state engine
Cfenjin is a parser and part of the Cfruby tool-set. It is made to look and behave like Cfengine, the GNU configuration engine, which was conceived in 1994 by Mark Burgess[1] and has a simple programming language that can be mastered by any qualified system administrator.
Cfengine was superior to other tools available at the time because it allowed a level of automating system administration while pursuing:
Readability
Documentation
Prevention of duplication
Abstraction
Cfenjin aims to be a better Cfengine. Like Cfengine it
is a simple language
has a class based decision structure, which documents policies
allows automation and abstraction
But in addition it
allows embedded Ruby can be relatively easily extended has extensive and configurable logging support
Instead of opting for the original Cfengine code base we decided to use the Ruby language as a natural parser.
Ruby is a fully object oriented scripting language[2] with a rapidly growing support base. Ruby lends itself very well to rapid programming and usually results in terse but readable code[3]. It's affiliation to Perl is close enough for most Perl programmers to quickly pick up Ruby. Furthermore Ruby is nowadays on most installed UNIX systems and runs on Microsoft Windows as well.
By using Ruby as a base language we get a very powerful scripting language for free. Besides being well tested, cross-platform and secure, it has a number of nice features out of the box (support for Perl-like regular expressions, arrays, hashes, a great syntax checker, libraries for XML parsing, FTP and LDAP support, YAML etc. thrown in) that have been listed on Cfengine's wish list for a long time.
Meanwhile Cfenjin users do not need to learn Ruby as Cfenjin is a self-contained language implemented as a pre-parser (the output is pure Ruby code).
Example 1: Classes & configuration
Cfenjin has classes and logical execution.
-
1 servers = ( screamer flightless ) 2 sshd = ( servers ) 3 4 sshd:: 5 6 $sshd_groups = 'sshlogin'
Here we tell Cfenjin that the servers are running sshd. If sshd is defined sshd groups gets set1.
Next we can use logic like
-
1 package 'ssh','openssh' 2 3 control: 4 5 if File.directory? '/etc/ssh' 6 @ssh_etc = '/etc/ssh' 7 else 8 @ssh_etc = '/etc' 9 end 10 11 files: 12 13 @ssh_etc+'/ssh_config' o=root m=0644 14 @ssh_etc+'/sshd_config' o=root m=0600 15 @ssh_etc+'/ssh_host_dsa_key' o=root m=0400 16 @ssh_etc+'/ssh_host_rsa_key' o=root m=0400 17 if File.exist? @ssh_etc+'/ssh_host_key' 18 @ssh_etc+'/ssh_host_key' o=root m=0400 19 end 20 21 editfiles: 22 23 EditFile.edit @ssh_etc+'/sshd_config' do | f | 24 f.ReplaceAllAppend "UsePrivilegeSeparation [Nn].*", 25 "UsePrivilegeSeparation yes" 26 f.ReplaceAllAppend "PermitRootLogin\s+[yY].*", 27 "PermitRootLogin no" 28 f.ReplaceAllAppend "PermitEmptyPasswords[[:space:]]+[yY].*", 29 "PermitEmptyPasswords no" 30 f.ReplaceAllAppend "^AllowGroups[[:space:]]+.*", 31 "AllowGroups #{$sshd_groups}" if $sshd_groups 32 f.ReplaceAllAppend "^AllowUsers[[:space:]]+.*", 33 "AllowUsers #{$sshd_users}" if $sshd_users 34 end
where we test where the configuration files are (note we are using plain Ruby in the control block). In the files block we check and set permissions and in the 'editfiles' block we edit the configuration file.
Note that editfiles allows for GNU type regular expressions, as well as Ruby and Perl type. Also the 'editfiles' block allows for 'if' blocks and other Ruby language syntax.
Example 2: Package
Cfenjin understands the package system. By using the 'package' command in a script it will skip configuration of a non-installed package. E.g.
package 'sudo' ... do 'sudo' stuff when package 'sudo' installed ...
this is why in Example 1 we do not need to surround every statement with an 'sshd::' clause(!)
Example 2: Tidy
Cfenjin has powerful tidying. The pattern search allows for full regular expressions. For example, to tidy all files older than 30 days matching:
tidy:
/var/log/ pattern='mail\.\d+\.gz' age=30
removes /var/log/mail.3.gz when older than 30 days.
1.3 Cfyaml, another state engine
If you want to use a code generator, or parse configuration files, a YAML format can be used instead (or you could implement an XML edition quite easily. For example the following may be a way of configuring your mailer installation:
- install: exim
- filename: /usr/local/etc/exim
options:
recursive: true
user: mailnull
group: mail
mode: u=rwX,g=rX,o-rwx
- filename: /etc/rc.conf
set:
- [sendmail_enable, '"NONE"']
- [exim_enable, '"YES"']
- [exim_flags, '"-bd -q5m"']
1.4 libcfruby, the library
By studying the rdoc generated help files you can get a taste for the number of classes and methods that are available to Ruby programmers.
Some quick examples:
-
1 # user and group management 2 os = Cfruby::OS::OSFactory.new().get_os() 3 @manager = os.get_user_manager() 4 grouplist = @manager.groups() 5 assert_equal(true, @manager.group?('bin')) 6 @manager.add_user("lijygrdwa") 7 8 # copying 9 Cfruby::FileOps.backup(@basefile.path) 10 Cfruby::FileOps.copy(copydirpath, newfilepath) 11 12 # tidying 13 Cfruby::FileOps.delete(@basedir, :glob => "^mail\.\d+\.log\.gz", 14 :recursive => true)
and, unlike standard Ruby library calls you get full and configurable logging. An extensive version of the log can look like:
cfruby info [2] copy /mnt/flash/darcs/cf/cfwrk/masterfiles/Xresources to /home/wrk/.Xresources - attempt cfruby info [3] copy /mnt/flash/darcs/cf/cfwrk/masterfiles/Xresources to /home/wrk/.Xresources - aborted - files have the same sha1 hash cfruby info [2] changing permissions of "/home/wrk/.Xresources" to "400" - attempt cfruby info [1] changing permissions of "/home/wrk/.Xresources" to "400" - done cfruby info [1] Computer "palermo" belongs to classes "any, debianlinux, laptop, linux, pacs, wrk"
And it is easy to tune it down to any level you want. For example you can choose to only log changes really made to the system. You can disable/enable parts of the logger and control where to send the output.
1.5 Where are we now?
The documentation still has some holes, and needs further elaboration and work. Nevertheless Cfruby as well as the library is rather complete, can be considered beta, and is being used in a number of production systems. Examples of Cfengine style scripts can be found in the ./documentation/cfenjin/examples directory. So if you are armed with the standard GNU Cfengine documentation and a Ruby manual you should be able to use Cfruby to your advantage.
1.6 Availability
Cfruby is open source and licensed using the GPL. Sources and documentation can be obtained from
2 References
[1] Mark Burgess. Cfengine, a configuration engine. ![[WWW]](/classic/img/moin-www.png){kind=small}
http://www.cfengine.org/.
[2] Yukihiro Matsumoto. The Ruby language. http://www.ruby-lang.org/en/.
[3] Pjotr Prins. Ruby: Productive programming language. http://www.linuxjournal.com/article.php?sid=5915.