How-To: Fix issues with VMware Server tools on Gentoo (and other distros)

VMware Server is a complicated beast. When it breaks, you’re usually without support (although paid support options do exist), and out of commission. One problem that tripped me up for a few weeks was that I had a running VM that refused to suspend. Because of this I couldn’t get clean backups of the VM in a suspended state and had to resort to just a plain backup of the filesystem via rsync. It worked, but it was less than optimal. Any time I ran the suspend command:

/opt/vmware/bin/vmrun suspend Gentoo.vmx

The error message I was presented with was:

Error: Command failed: Cannot connect to the virtual machine

I could hobble along like this for a while but today I needed to add another virtual disk which required a restart of the VM. When I tried to start it via the command-line tools like this:

/opt/vmware/bin/vmrun start Gentoo.vmx

The system just hung indefinitely. Eventually I had to CTRL-C out of it and get back to the command-line to figure out the issue.

What was the underlying problem? Almost every time I’ve had a VMware Server issue it has been permissions. When you’re working with VMs you should always make sure that the following criteria is applied to the files in the VM’s working directory:

  • All files must be owned by the vmware user and/or the vmware group
  • All of the files must be readable and writeable by the vmware user and/or group
  • Disk files must be marked as executable by the vmware user and/or group
  • Optional: All permissions should be removed from anyone other than the owning user and/or group

What I do to get this done quickly on all of my VMs is run the following few commands in each VM’s working directory:

chgrp vmware *
chmod 660 *vmem
chmod 660 *.vmsn
chmod 750 *.vmdk
chmod 750 *.vmsd
chmod 750 nvram

After running those few commands VMware Server works like a charm again.

How-To: Setup streaming MP4 video on your own Gentoo server (and other distros)

With so many free streaming services available today (Vimeo being the best and YouTube being the most widely recognized) you might wonder why someone would want to host their own videos. My main reason is so that I can more easily see who watches my content. Other reasons include better access control if your videos are personal in nature, or a more integrated, branded user experience if you’re producing content that is subscription, pay-per-view or ad supported. You might also want to get around the restrictions that come with the free accounts these sites offer without paying for their premium services. If you’re already invested in the hardware to host your site and you can support the bandwidth requirements for your expected traffic volume then it might make sense to host things yourself.

So, what’s the big deal? If you’re looking to get started right away you can render an MP4 file, install a free player like FlowPlayer, and start hosting video in 5 minutes. The problem that you’ll notice straight away is that your videos have to buffer completely (ie. the user’s browser downloads the whole video) before they start playing. For short videos or video files that are small this isn’t a big deal. But for HD videos and videos that are long in duration the delay can cause even the most patient viewers navigate away from your site. Since all of the other services have gotten users accustomed to video playing immediately it’s hard to justify this kind of behavior on any site today.

I’ve figured out the secret sauce to get this working. Now you can do it too with just these few simple steps:

  • Step 1: Render your MP4 files and place them on your server
  • Step 2: Create a webpage with FlowPlayer embedded. A template to get you started looks like this:
    <head>
    <script type="text/javascript" src="http://your-site.com/flowplayer-3.2.4.min.js"></script>
    </head>
    
    <a href="http://your-site.com/your-video.mp4" style="display:block;width:800px;height:475px" id="player"> </a> 
    
    <script>flowplayer("player", "flowplayer-3.2.5.swf",
    
    { clip:
      { url: "http://your-site.com/your-video.mp4",
        duration: 120,
        metadata: false,
        autoPlay: true,
        autoBuffer: true,
        bufferLength: 3,
        linkUrl: "http://your-site.com/your-video-description.html"
      }
    });</script>

    Just replace your-site.com/your-video.mp4 with the real URL of your MP4 video and your-site.com/your-video-description.html with the site you want to link the user to if they click on the video while it’s playing. Also, replace “120″ after the duration variable with the actual duration of your video in seconds. If you don’t know this value, or you’re feeling lazy, you can set it to zero and FlowPlayer will figure it out for you. Don’t set it too short or your video will end prematurely and make sure that flowplayer-3.1.4.min.js is on your server in the appropriate location. This example assumes it is in the root directory.

  • Step 3: Download and install mod_h264 for Apache. There are instructions for distros other than Gentoo or youcan run these commands on Gentoo to do it all:
    cd /tmp
    wget http://h264.code-shop.com/download/apache_mod_h264_streaming-2.2.5.tar.gz
    tar xzvf apache_mod_h264_streaming-2.2.5.tar.gz
    rm apache_mod_h264_streaming-2.2.5.tar.gz
    cd mod_h264_streaming-2.2.5
    ./configure
    sudo make install
  • Step 4: Add mod_h264 for your Apache configuration. Append these lines to the bottom of your /etc/apache/httpd.conf:
    LoadModule h264_streaming_module /usr/lib/apache2/modules/mod_h264_streaming.so
    AddHandler h264-streaming.extensions .mp4
  • Step 5: Restart apache with
    /etc/init.d/apache restart
  • Step 6: Test out your streaming video. If all went well then you’ll see the bar behind the playhead fill in as the video buffers.

That’s all there is to it. Thanks to the mod_h264 developers (and these instructions) you’re now able to stream MP4 video like the big guys. Here’s an example of what it should look like (the source material in this video is from FreeStockPhotos.com but the controls in the player cut off their watermark):

How-To: Disable screen blanking in X on Gentoo (and other distros)

Even though I have a nice dual monitor setup in my office I tend to use my laptop for almost everything. The dual monitor system ends up holding a bunch of diagnostic and performance information (top, PProcM, tail of application/database logs, etc) about the various machines I’m working on. Because of this I’m almost never using the keyboard and mouse on that machine and in fact it often doesn’t have a keyboard and mouse hooked up. Typically I just access it just using x11vnc so I don’t have to switch my wireless mouse over or go find a free keyboard.

The problem is that the screen blanker kicks in every 15 minutes or so. I got tired of reconnecting via VNC to turn the displays back on so I just did a quick mod to my /etc/X11/xorg.conf and now I just turn the displays off manually whenever I don’t need them. If you want to do the same thing just add the following section right before the input device section (found on these forums):

Section "ServerFlags"
    Option         "blank time" "0"
    Option         "standby time" "0"
    Option         "suspend time" "0"
    Option         "off time" "0"
EndSection

How-To: Fix VMware Workstation’s modules after updating your kernel on Gentoo

Here’s another thing I ran into today when adding support for KVM to my kernel… VMware Workstation’s modules failed to load at boot time. Normally, the fix is to run:

vmware-modconfig --console --install-all

Or:

emerge --config vmware-workstation

Neither of these will work on the latest versions of Gentoo. The real fix is to re-emerge your vmware-modules package. You can do this by just running:

emerge vmware-modules

That will build the modules against your current running kernel. After that, your modules will work again but will not be loaded. You need to forcibly reload them by running:

/etc/init.d/vmware restart

Each time you change your kernel, which means even booting into one that previously worked, you’ll need to do this again. For convenience if you change your kernel a lot you can just put those two commands into a script. The script, which I call “fix-vmware-modules.sh”, should just look like this:

emerge vmware-modules
/etc/init.d/vmware restart
rc-update add vmware default

I added the extra rc-update command just in case you run this on a machine that you just installed VMware Workstation on. It makes sure the modules are loaded at boot time and it won’t hurt an existing installation. Instead it will just let you know that it’s already added to the default runlevel.

How-To: Fix “configure: error: C compiler cannot create executables” on Gentoo

I encountered this message today when trying to update glib-2.20.5-r1 on my 64-bit Gentoo box. I’ve seen this error message before but today, as luck would have it, I had just made some changes to my make.conf so I knew exactly where to look. What I was doing was preparing my system for Xen and I was duped into adding the “–mno-tls-direct-seg-refs” option to my CFLAGS. It turns out that 64-bit Gentoo doesn’t need this option and actually it will cause your C compiler to fail.

If you have this option and you’re trying to get Xen running, remove it and try again. That will most likely fix your problem. If the problem persists you can dig a bit deeper and figure it out like this.

  • Step 1: Scroll back through your emerge error. If it’s too far gone then retry the emerge and get it to report the error again. If you’re using screen don’t forget that you can use CTRL-A and then left bracket to enter copy mode and maneuver through the scrollback buffer. Press escape to exit it when you’re done. You’re looking for something that looks like this:
    configure: error: in `/var/tmp/portage/dev-libs/glib-2.20.5-r1/work/glib-2.20.5':
    configure: error: C compiler cannot create executables
    See `config.log' for more details.
    !!! Please attach the following file when seeking support:
    !!! /var/tmp/portage/dev-libs/glib-2.20.5-r1/work/glib-2.20.5/config.log
  • Step 2: Open the file residing at the full path of the config.log file (in my case /var/tmp/portage/dev-libs/glib-2.20.5-r1/work/glib-2.20.5/config.log)
  • Step 3: Search the file for the string “failed program”. A few lines up from this message you’ll see the true error. In my case it was:
    configure:3385: x86_64-pc-linux-gnu-gcc -O2 -pipe -fomit-frame-pointer --mno-tls-direct-seg-refs   -Wl,-O1 conftest.c  >&5
    cc1: error: unrecognized command line option "-fmno-tls-direct-seg-refs"
  • Step 4: Remove the unrecognized command line option. It is reported here as “-fmno-tls-direct-seg-refs” but in the make.conf file it is ”–mno-tls-direct-seg-refs”. You’ll see that in the line above the error message. I’ve left it in there so you can see what that line should look like too. Your error may be different entirely or just a different flag. Either way, fixing the issue reported there should bring your compiler back to life and keep you emerging in the future.

Tip: Get VMware Workstation 6.5.3 working on Gentoo

The final piece to my reinstall was to get VMware Workstation 6.5.3 running so I could use my nice Linux box (8GB RAM, fast RAID configuration) to run virtual machines instead of running them on my much less powerful MacBook Pro.

To do this installation you need to download the files from VMware directly, move them into portage, then do the installation from a real console (SSH or screen sessions won’t cut it). That was no problem but I ended up staring at a message like this

process XXXXX: Attempt to remove filter function 0xXXXXXXXX user data 0x88162a8, but no such filter has been added
  D-Bus not built with -rdynamic so unable to print a backtrace

I tried to re-emerge dbus with that option but there are no USE flags that correspond to it that I could see. I checked if D-Bus was even running and it was. At this point I randomly and luckily found the fix. All you need to do is to stop D-Bus via

/etc/init.d/dbus stop

As soon as I did that VMware started up, asked me to accept the EULA, and I was straight back on the virtualization track.

Note: While stopping DBus will get VMware to start you’ll still need to do two last things to make everything happy

Step 1: Run

/etc/init.d/vmware start

Step 2: Run

rc-update add vmware default

This makes sure that VMware creates all the device nodes that it needs, loads all of the modules for virtual networking, monitoring, etc., and that this will always come up after a reboot. Without this you can end up chasing down some seriously weird error messages.

How-To: Fix “ImportError: No module named WebappConfig.config” on Gentoo

I just tried to upgrade trac on my Gentoo box to get back into the swing of tracking and squashing bugs. Gentoo was having none of it and reported “ImportError: No module named WebappConfig.config”. That error by itself isn’t terribly helpful so in order to fix it I determined you usually need to do a few things. If you aren’t installing trac, replace trac in these steps to the application you are trying to install.

Step 1) Run

python-updater

This can take a LONG time so be ready to wait.

Step 2) Try to run trac. If that doesn’t work, go on to step 3. If it does, you’re done.

Step 3) Try to emerge trac again. If that doesn’t work, go on to step 4.

Step 4) Run

emerge webapp-config

trac should be updated by python-updater but the three other steps are just a backup to be extra sure that everything is installed properly. After jumping through those hoops you should be back online again.

How-To: Fix the “cannot find -luuid” error on Gentoo

Recently the Gentoo project deprecated lots of system profiles. What this meant for me what that I needed to do nearly an entire system rebuild with my usual “emerge –update –deep –newuuse -av world” magic. After churning through about 200 of the 400+ ports that needed to be rebuilt my system started getting weird. Notably mount/umount stopped working, and the emerge failed. The error message I got after a bit of digging was that Gentoo “cannot find -luuid”. Running mount or umount (and probably other e2fsprogs utilities) gave me the error “error while loading shared libraries: libblkid.so.1: cannot open shared object file: No such file or directory”.

When you get an error like this it’s time to act fast since a power outage or a reboot can really hose the system. I’ve had it happen before and it’s not pretty. In this case you need to re-emerge e2fsprogs and util-linux. I simply did this:

emerge --update --deep --newuse -av e2fsprogs

That re-emerged both packages for me. If it doesn’t show that it will re-emerge both then you’re best off doing this:

emerge --update --deep --newuse -av util-linux e2fsprogs

After doing that you should have neither “cannot find -luuid” nor “error while loading shared libraries: libblkid.so.1: cannot open shared object file: No such file or directory” problems. If you’re extra paranoid like me you should also re-emerge dbus and then do a revdep-rebuild

emerge --update --deep --newuse -av dbus
revdep-rebuild

If you don’t have revdep-rebuild installed you can install it by running

emerge gentoolkit

dbus tells you that you must run revdep-rebuild after emerging it and restart dbus. Listen to it, it’s not kidding.

How-To: Resolve PyUnicodeUCS2_DecodeUTF8 errors in Gentoo

Just yesterday I was doing an upgrade of my Tomcat installation (and nearly everything on my virtual machine for that matter) and re-emerging the Tomcat servlet API kept giving me the error “unresolved symbol PyUnicodeUCS2_DecodeUTF8″ and told me to re-emerge pyxml. It turns out that pyxml is fine and thanks to a lone comment on the Gentoo bug tracker I found a solution. Gentoo may break from time to time but their bug tracker is truly gold when you hit upon the right bug and the right comments.

Step 1: Run “emerge --sync

Step 2: Update your packages as usual (I run “emerge --update --deep --newuse -av world” but people tell me I’m crazy, it can break LOTS of things if you don’t know what you’re doing)

Step 3: Wait for the unresolved symbol error to halt the emerge process and then run “emerge -1 javatoolkit

Step 4: Repeat step 2 and you should be in the clear

How-To: Build libwiimote-0.4 on Gentoo

This may only be related to 64-bit Gentoo but if you’re having build troubles try starting from scratch like this. Note: These are all commands that you’ll have to run directly. There is narrative except in step 6.

Step 1: wget http://downloads.sourceforge.net/libwiimote/libwiimote-0.4.tgz?use_mirror=internap

Step 2: tar xzvf libwiimote-0.4.tgz

Step 3: cd libwiimote-0.4

Step 4: autoconf

Step 5: ./configure

Step 6: Edit src/Makefile and add “CFLAGS=-fPIC” to it. If CFLAGS already exists just append ” -fPIC” to it.

Step 7: make

Step 8: make install

Now you’ll have libwiimote-0.4 built and installed. All you need are a few applications to take advantage of it. Stay tuned.

Mobile Analytics by Mixpanel