Life Lessons
1. Read the manual.
2. Read the manual.
3. Don't trust the manual.
While most hardware and software vendors provide some form of documentation with their products, more often than not we fail to read it _before_ we use the product. Unfortunately, many times the documentation fails to mention certain items or has various "mis-statements".
As the late US president Ron Reagan used to say: "Trust, but verify."
Wednesday 23 April 2008, 12:55 AM
PHP5 - MIME type quirks
It has been a while since I did any web development, and I recently decided to get back into PHP development. After downloading the latest version of PHP for my IIS server I proceeded to do some tinkering just to get back into the feel of the language. Just for grins I decided to figure out how to send a directory listing to the browser, adding links for any image files that were found so that the user could click on them and display the image in their browser. Simple, right? Well apparently not. I must have spent about 4 hours last night on Google trying to find out why the MIME typing was not working correctly, nor the various PHP extension libraries being loaded properly. I finally managed to figure it out this afternoon:
PHP comes with two extension modules for determining the MIME type of a file: Fileinfo and Mimetype. Both require a mime.magic file that stores the MIME type definitions, but they access it in two different ways. Fileinfo requires that the path to the magic file be passed to the finfo_open function. Mimetype, on the other hand, uses a setting in the php.ini file that identifies the magic file.
Now the only reason why I mention anything about this topic is that judging from the plethora of Google hits I came across while trying to solve my dilemma I realized that this is not well-documented anywhere, not even in the PHP documentation. So here is a concise explanation:
The MIME magic files
If you go to the GnuWin32 project page on SourceForge you can download the MIME magic files and unzip the archive to your PHP installation folder.
mkdir C:\PHP\magic
... unzip the MIME files into C:\PHP\magic
should now contain:
magic
magic.mgc
magic.mime
magic.mime.mgc
Using Fileinfo
In your php.ini file, make sure that the DLL gets loaded automatically by PHP:
... add to the end of the file where other extensions are loaded
extension=php_fileinfo.dll
Now within your PHP file you can do the following:
$finfo = finfo_open( FILEINFO_MIME, "C:/PHP/magic/magic" );
$mimeType = finfo_file( $finfo, "C:/PHP/php.gif" );
finfo_close( $finfo );
echo $finfo;
.... should display image/gif
Using Mimetype
First, make sure that the extension is loaded and that it knows where the magic file is located.
mime_magic.magicfile="C:/PHP/magic/magic.mime"
extension=php_mime_magic.dll
Now in your PHP page you can do this:
echo mime_content_type("C:/PHP/php.gif");
... again, image/gif
That's all there is to it. Now, a couple of things to mention ... the Mimetype extension is marked as deprecated with the Fileinfo extension being the preferred method for determining file MIME type. I'm not sure why this is the case, as for me it makes more sense to have the MIME magic files be a system-wide setting, but according to the PHP extensions documentation Fileinfo is the way to go. The other thing to note is that Fileinfo uses the C:/PHP/magic/magic file whereas Mimetype uses C:/PHP/magic/magic.mime. It is very important to get these right. If you switch them up it will not work. I tried this to make sure that that is the case and was unable to get the MIME type.
For anyone out there who does regular PHP development, you all probably knew most of this already. For all those who are new to PHP or re-learning it like me, I hope this helps clarify and resolve any problems you have encountered with this set of extensions.
Monday 21 April 2008, 6:21 PM
Replacing the SS4000-E: Part I
Well as I mentioned in my earlier post I am trying to find a way to replace my SS4000-E as our production file server. To that end I have begun to build a Linux From Scratch server, upon which I plan on adding just the necessities to have a working file server in our Windows LAN. I was hoping to have a new system to work with that was complete with SATA RAID, a gigabit NIC, lots of RAM, etc, but I decided to work off of one of my old PCs first as a practice run and work out all the snags before building the final production box.
In the course of this past weekend I was able to build the basic LFS system and have already added some security software, as well as some simple utilities, but that has been about it thus far. (I figured it was time to get some sleep after 4am this morning) If all goes well I should have the rest of the pieces put together by the end of the week, and come next Monday I will be able to throw it to the wolves and have our microfilm scanners try and beat it up for a few days. I will be surprised if the performance does not meet my expectations, even for a setup that is by far scaled down as compared to the final system I plan to use.
Just for those who are not familiar with LFS, I suggest you take a look at the Linux From Scratch website. The project is lead by Gerard Beekmans, and he and his group of developers have done an excellent job of outlining the entire process of creating a Linux system completely from source code. It is not a quick process, but it really is not all that difficult either once you have read through the LFS documentation. This is the third or fourth system that I have built using their procedures and the others had served well for a long time. It just takes patience and a decent knowledge of Linux.
The other tool which I suggest all systems administrators take a look at is Knoppix. This is a Linux distribution that runs entirely off of CDR and makes a great recovery tool, even for Windows-based PCs. In building my LFS box I used this as my base Linux system, allowing me to run off of CDR while I built directly onto my hard drive.
Once the final system is in place I'll post back a list of the software installed so that anyone else who is interested in building one of their own can replicate the procedure. Also, I invite comments on which software choices should / should not be included and why. I am curious to see what other admins use on a regular basis in administering their servers, and any tips / tweaks that they have found.
Tuesday 15 April 2008, 6:57 AM
Intel's SS4000-E - so you're having issues with one too, eh ...
About six weeks ago we realized that we needed more production storage space and decided to look into a NAS solution instead of a traditional server. After doing some searching I came across a neat-looking little black box made by Intel, their SS4000-E. I read through the specs and called up TigerDirect to get some of my questions/concerns answered, the primary question being "will it handle the throughput we require each day?" We are a document conversion business that digitizes paper and microfilm/microfiche, and on any given day we may scan well over 500K images, each at around 65K as the average file size. Needless to say, we planned on beating the devil out of this storage box. At any rate, TigerDirect told me that yes, the system will be able to handle the throughput without any problems. Ok, that's great. So we ordered one ......
Well, I wish I had found some of the postings I have read since then describing the "shortcomings" of the system. I realized, much to my chagrin as CTO and resident tech guru, that my purchase was not all it was sold to be. Part of our processes involves building multi-image TIFF files, and some of these files can be upwards of 15-25MB in size. I realized immediately that something was awry when I tried processing out to the box; the system came to a screeching crawl. Come to find out later that part of the issue was with the drives that we received with the unit. For those who don't know, it comes with a set of SATA-300 drives. On the back of these drives (something I had not thought to check as I have only used EIDE drives to this point) there is a jumper that limits the throughput of the drive to 1.5Gbps instead of allowing the full 3.0Gbps data transfer rate. This apparently was installed for those cases were the motherboard did not support the 3.0Gbps rate. What made no sense to me is that this system states that it does support the faster transfer rate, so why put the jumper on there?
So ... off came the jumpers. Ok, the transfer rate sped the system up from a crawl to a slow walk. Obviously something else is still wrong. Being the Linux "guru" I decided to try and access the system directly via SSH and see what was going on. One would figure that the box, being a Linux-based system, would have a SSH daemon running, right? Nope. Oh, and don't waste your time looking for a way to turn it on in the web configuration interface. I had to do some Google searching to turn up the solution to this one. There is an undocumented CGI script that gives you the ability to turn the SSH daemon on, but you have to put it into your address bar yourself. None of the other CGI pages will take you there.
Ok, now I have my SSH access ... great. I get in there and find out that (a) you're stuck with XFS as the file system on your NAS shares and (b) apparently no one thought that turning on the noatime mount option was a good idea. The mounts are not listed in the fstab, either. There is a conf file that you have to hunt for, add the noatime option, then reboot the box. After all that, it now is no longer at a slow walk, but rather a light jog. (sigh)
If there is anyone out there that can enlighten me on how to speed up the XFS file system, I would appreciate the advice. I have worked with reiserfs in the past and had much better results. Unfortunately the box does not come with reiserfs installed (although according to the conf files it is supported), and even if it were there is no way that I have found to tell the box to make a new share using anything other than XFS.
The only consolation I have found is that there are a number of others out there who also have similar problems with their systems. I even saw an article earlier that discussed a way to install a Debian distro onto the box, but the steps required seem daunting at best. One would figure that Intel would put out a better product than this, but I also have found out that it was not their product in the first place; they bought it from some other manufacturer and have basically no way to support the internal workings of the system.
So, what to do with it? It does make a good storage solution, meaning as long as you write the data once and do not require any substantial processing to be done to that data once it is there, it will work well. It is stable and will run probably for months at a time, if not longer, between shutdowns for maintenance. It just is not designed to handle massive amounts of reading and writing all at one time. At this point, that will be its ultimate mission in our shop, but it sill leaves us with the problem of not having enough production workspace.
To that end, I have decided to take a more traditional approach and install a Linux-based file server. As cost is an issue, I have been pricing out some bare-bones solutions upon which I can build. Also, I have decided to build a Linux From Scratch OS onto the box, instead of installing a heavy distro. I like Mandriva and Ubuntu, but I really do not want to have all that extra software and package management to deal with. I was able to work out a list of applications/libraries that I will need, cutting out all of the fat and trimming the system down as much as possible. Once I have gotten everything successfully installed I will post my results and any performance issues/benefits that I encounter. If all goes well, I should have a solution for anyone else out there who bought one of these storage boxes and is looking for a higher-performance solution.
