June 30, 2009

A Method for Migrating Files and Permissions

I recently had to retire an old Solaris system. It was the one that had a decade-worth of random projects placed on it. This leads to lots of files that can be a headache to handle without a skilled team and ticketing system to back you up. My part of the project was migrating the web content. To follow is a script that automated the copy of the web directory and mapped old UIDs to new UIDs, with non-colliding GIDs preserved. This does not include the 1000 line Apache httpd.conf file I had to audit and update.

Everything went surprisingly smooth, with only a handful of special cases to deal with, and a nice log file to catch those anomalous instances. New accounts were not created, but instead orphaned and chowned to a dummy user account.

Notice that I am using read into a while loop instead of a for loop from find. This is a good way to handle filenames with spaces and special characters.

migrate-www.sh:
#!/bin/bash

### uncomment afterwards and do not re-run
#exit 1

### sync files from old to new host
mkdir /www-old
rsync -aq old_server:/www/data /www-old/

### copy old auth files and manually edit groups
rsync -aq root@old_server:/etc/passwd /www-old/passwd.old
rsync -aq root@old_server:/etc/group /www-old/group.old
cat /www-old/group.old >> /etc/group
vim /etc/group

### map old UIDs to new UIDs
#for I in `find /www-old/data`; do
find /www-old/data | while read I; do
echo -n "$I"
OUID=`ls -ndl "$I" | cut -d ' ' -f 3`
echo -n " OUID $OUID"
OLOGIN=`grep "\:x\:$OUID\:" /www-old/passwd.old | cut -d ':' -f 1`
echo -n " OLOGIN $OLOGIN"
NUID=`grep ^$OLOGIN\: /etc/passwd | cut -d ':' -f 3`
if grep -q ^$OLOGIN\: /etc/passwd; then
echo " NUID $NUID"
chown $NUID "$I"
else
# chown jack(1209) files to john(3572)
case "$NUID" in
"1209" )
echo " ADOPT 3572"
chown 3572 "$I"
;;
# own jill(921) to jane(372)
"921" )
echo " ADOPT 372"
chown 372 "$I"
;;
*)
echo " ORPHAN 499"
chown 499 "$I"
;;
esac
fi
done

### special case perm changes
# adopt to bob
chown -R 5432 /www-old/data/projectA
# new GID for group 432 is 1487
find /www-old/data -gid 432 -exec chown :1487 {} \;

Execute the scipt and redirect all output to a log file for later analysis:
# chmod +x migrate-www.sh
# ./migrate-www.sh > migrate-www.log 2>&1 &
# tail -F migrate-www.log

June 19, 2009

Fix the Domain of System Mail

If your system sends you reports via email with the sender root@localhost.localdomain, there is a quick fix. add the following lines to /etc/mail/submit.mc subsituting your own hostname:
define(`confDOMAIN_NAME',`yourhost.yourdomain.net')dnl
define(`always_add_domain')dnl

then reload the config and sendmail daemon:
# cd /etc/mail
# m4 submit.mc > submit.cf
# service sendmail reload


Thanks, Dareus.

Grid Engine Job Prioritization

Figuring out how to balance the sharing of a computational cluster between groups and projects can be a bit of a headache. The difficult part is the logistics of defining real world resource sharing from a business perspective and then implementing them within your queue configs and code. Here comes Sun Blueprints to the rescue. Charu Chaubal has written an excellent and understandable paper on Scheduler Policies for Job Prioritization in the N1 Grid Engine 6 System. Thanks, Charu. This has helped me out enormously.

June 10, 2009

Fedora 11 in the Wild

I got a big kick out of the Fedora 11 release announcement this morning. The announcement is done up in pith helmet, explorer club fashion, as if Fedora is a wild beast from some far-off continent. As you may know, most if not all of these new features will eventually make it into enterprise Linux. Give it a read.

May 22, 2009

Drupal Security Embarrassment

Recently, Justin disclosed a new cross site scripting vulnerability found in Drupal. Justin is a security consultant and programmer who contributes to open source projects with his expertise in web application security. Drupal is an open source software project that enables the deployment of web portals, blogs, content management systems, and really any web-based content. The response to the disclosure is embarrassing for the entire Drupal project and bad for its reputation.

Drupal has been adopted by many public-facing organizations for its web presence. Now the software project is feeling pressure due to the transition from small user group application to enterprise grade CMS. One notable company supporting Drupal commercially is Aquia, with an impressive list of partners and clients. That being said, there are more people relying on the code in larger organizations with tiered web content delivery, where user permission problems and security exposures are crucial to ferret out and handle properly. So you would think the Drupal developers would graciously accept and respond favourably to feedback on security issues. Not so.

When Justin approached the Drupal Security Team about the recently discovered vulnerability, they refused to correct the problem. When a software project refuses to patch a flaw, I agree with the full-disclosure route. People need to know what the risks are and make educated and informed decisions. After Justin went full-disclosure, the Drupal Security Team proceeded to personally attack him. One of the team members, Karoly Negyesi, even made a veiled threat which included publishing Justin's address.

The following was posted by Karoly Negyesi 1drupal.blogspot.com lists.drupal.org:
This guy believes in full disclosure so much he discloses everything
he finds instead letting us fix and disclose. This happened more than
once. So surely he wont mind if I disclose his mail sent to the
security list. According to whois, he is

Justin Klein Keane
XXX XXXXXXX Street
Philadelphia, PA 19123
US
Phone: 1-215-XXXXXXX
Email: jkeane at XXXXXXX.net

I will let the creative members of the Drupal community figure out
ways to express their displeasure with his practice. Mail follows:
...


This is the most embarrassing and unprofessional stunt I have ever witnessed in software development. This borders on criminal. You don't threaten someone's personal safety because you don't like his contribution to the project or disagree with him.

Karoly Negyesi has just placed a black mark on his character and career. I hope that the Drupal community will take corrective action and remove this person from the project, lest it also give Drupal a bad name.

Read more about it on Justin's blog.