Apache and HTTP Headers with Underscores

Starting with Apache 2.4, headers were dropped that contained items to include underscores and expects dashes instead. Red Hat backported this change into the Apache 2.2 that ships with 6.7. You can read more here:https://httpd.apache.org/docs/2.4/env.html

If you run into the issue where your REMOTE_USER or similar is not being passed to your WSGI application or similar, it is most likely due to the above. Good news, easy fix. I came across this issue with the wonderful Oracle Access Manager (OAM) 10g. Side note: what a crappy piece of code and a disgrace to authentication mechanisms everywhere.

In this example I am looking for OAM_REMOTE_USER.

In your virtualhost config:

SetEnvIfNoCase ^OAM.REMOTE.USER$ ^(.*)$ fix_accept_encoding=$1
RequestHeader set OAM-REMOTE-USER %{fix_accept_encoding}e env=fix_accept_encoding

If you are using mod_wsgi for Django or Flask, you will need to add:

WSGIPassAuthorization On


Lenovo Yoga Tab 3 Pro Review

After the unfortunate loss of my 2014 Nexus 7 tablet I was on the prowl for a replacement. I prefer to keep an Android tablet due to the flexibility to use it as a mini Linux workstation if the need arises. The USB OTG support means the tablet is capable of taking on most of the tasks I use my Linux laptop with in the event I am stuck somewhere. Additionally I wanted a tablet that really excelled at media consumption as that is the primary use for a tablet in my workflow.

  • Great battery life
  • Android based
  • Media centric
  • USB OTG Support
  • SD Card support12556023_539159696241066_1175203354_n

Truth be told I prefer iOS-based devices for the security aspect, but I remove all sensitive and work related accounts from my tablets and accept Androids still broken security model.

I ended up looking at the latest and greatest from each vendor and ruled out most of them due to lacking some part of my requirements list. The one that really piqued my interest was the Lenovo Yoga Tab 3 Pro due to the projector, integrated stand, and great battery life. I had a heck of a time tracking one down stateside, but was able to order one directly from Lenovo.

Now having traveled with the device (the true test of a tablet) I feel I can offer my overall feedback on the device.

The good

  • Battery life is easily the best for screen-on I have seen in a tablet. The iPad has MUCH better standby, but for watching TV shows and Movies it is hard to beat
  • I like the hand grip more than I expected
  • The projector is enough for watching TV shows while we fall asleep in hotel rooms. We learned from this trip though we need a better mechanism to stand it up in odd-shaped hotel rooms
  • The SD card works as advertised and we brought tons of media
  • Lenovo didn’t muck up the Android build with too much crap, but they did severely limit the device
  • Screen is beautiful and the front-facing speakers are incredible. Why any tablet ships without front-facing speakers is beyond me

The bad

  • Slow and jerky. I kid you not. It makes using an iPad an absolute joy compared to Android. 2GiB RAM appears to be the biggest hinderance, but I blame it more on the Lenovo kernel build (~800MiB in use after fresh boot) as well as Android’s piss poor memory management. I type my pin in wrong all the time due to the numbers not registering
  • Android 5.1.1 is tough to use after having Android 6. It appears that Lenovo will slowly update this device as they do all others. They have released a series of patches the past month, but none that actual fix the systemic issues
  • No accessories as it isn’t a Samsung, Apple, or Google device. Lenovo needs to build an ecosystem around their flagship devices with simple things like cases and similar
  • The screen is incapable of staying at full brightness while playing intense games like Atlantic Fleet which leads me to question what the QA process was for the device
  • It feels…cheap…the stand creaks and the edges don’t align as much as I would have hoped for a device at this price point

Overall I’d say unless you really want a projector and stand built-in; buy an iPad. I keep hoping that Google will fix their atrocious tablet OS and hoping Android vendors actually focus on delivering a quality product versus churning out SKU. If it were not for my geekiness and needing to run dual mobile OS for things I would not be tossing my dollars at them until they can get their heads on straight.

Links on the issues with the tablet:




You are “dev” to me


The idea of keeping a production and development network always wears on me. Unless the change management  is in place to offer some assurances that both threads are kept exactly in sync, they inevitably turn into a game of “not good enough to buy down risk.”

That was the rationale behind creating a system that allowed rapid captures of running machines, transitioning to a closed network, and with no changes to the network/systems have them be as they were just minutes earlier. Thanks to technology from EMC, Cisco, VMware, and VyOS this is all very possible.

Enter “Test Network”

The number of VLANs and separate network complicates things. The goto configuration would be to simply generate a new VLAN and dump the machines into it, but what happens when the machine you copying over expects routing in place?

The entire point of test is to reduce the risk that your changes will have adverse effects. Your test configuration should always have sufficient fidelity to mitigate the level of risk you and leadership are willing to accept.

Yippie for software routers! Yeah for Linux!

Vyatta unfortunately got gobbled up by Brocade a few years ago. Luckily a very talented team continued forward with the last open-source version and have marched forward VyOS.

Using VyOS to mimic an expansive Cisco vSwitches/Routing architecture allows you to not worry about the rest. A single VLAN can be spun-up to keep it “physically” separate while attaching sufficient interfaces to the VyOS box to make it the new core.

The process then becomes (either HITL or scripted) the below:

  1. Identify the systems to test
  2. Take latest VMware Snapshot or EMC Avamar in this case and deploy BMR on machine attached to VLAN of Test Network
  3. Deploy peripheral machines for support
  4. Start!

Simple Diagram Overview


Blank Flowchart - New Page

Lessons Learned

Some things to keep in mind if you decide to move forward with a similar configuration:

  • Be aware of software licensing before moving systems. Ensure the vendor provides support for testing with the license you now run.
  • Servers are often supported by peripheral services such as AV, AD, and others. Be sure for a clean environment you include those in your scripts for copying over.
  • VMRC helps a ton in Vsphere 5.5 and higher, but in lieu of this ability you will need some means to transfer files back and forth.
  • 10 NIC is the most you can attach to a VM. This will complicate things with VyOS. Keep in mind you only need the 10 networks you are using and not ALL of them at once. So A, C, E, F instead of A-F.
  • This can be replicated a thousand times over with only your capacity being the hinderance. Test Network 2-7 are okay too. Just replicate the configuration.
  • Active Directory Tombstones will bite if you are not careful. 60 days and you should plan on dumping and starting fresh.
  • Nothing stipulates you cannot migrate a Test Network to Prod. Keep in mind that in this method the copy goes both ways.
  • Systems with external interfaces will need simulators to mimic those connections. Especially important in manufacturing scenarios.
  • System load is usually a big impact point to keep in mind when testing configurations. Be sure to include ways to force system load to match enterprise. Maybe I will write a blog post on the various SQL, HTTP, SSH, etc. ways to do this with some python code..


  • You are guaranteed to have a test network that represents your production network.
  • The on-demand nature ensures that storage and CPU cycles are only used when needed.
  • Depending on the size of the machine this can take any where from 30 minutes to 2 hours, but it is quicker than spinning a new environment.
  • Serves as a great disaster recovery test.


Liquorice Stick and Memories

“Mom, I don’t want to play clarinet! It is for girls!” I remember yelling at my Mother as I prepped for band my middle school years. We were poor growing up and the only option I had to afford an instrument and be in band was to use the hand-me down clarinet. My Grandmother had spent a decent amount of money when my Mother was the same age as I was then on a Buffet Crampon. For whatever reason I had associated some sexuality to all wind instruments and as such considered this suggestion from Mom an attack on my very blossoming manhood.

I never truly embraced the instrument until a visit to Tennessee. I suppose my Mother had spoken to my Grandfather or maybe I had mentioned it, but he proceeded to ask me about my liquorice stick playing. “Liquorice stick?” I had never heard my female centric clarinet called that. “Oh yes, Benny Goodman played a liquorice stick like no ones business.” Grandpa Hardie then proceeded to play a few big band songs where my much despised clarinet was central to the song. I had never heard music like that and I immediately latched onto it. My Grandfather gave me a love for the music and an identity for this instrument I never would have found on my own. He taught me that day, and many others, that being a cool guy was more about owning the role than what others thought about it.

There are countless other memories of my Grandfather I could share. Westerns, Cheezits, growing up in the hills of Kentucky poor, etc. But I will always cherish knowing my Grandfather was the guy who instilled in me a sense of charm and wit. He had a certain demeanour that always oozed a sense of calm and untroubledness that I have attempted to replicate in my adult life.

I have struggled to put in words the feelings of losing him this week. One of our last lucid conversations was him telling me about his life being complete. “I have done everything I wanted and I am ready.” He knew more than I did his time was coming to a close. He never wanted some big thing on his passing. His wish was for everyone to be happy and joyous for the amazing life he lived. No man is without faults or shortcomings, but I hope I can replicate, in addition to his composed charm, being able to close that chapter of my life in the same dignified and polished manner.

I bought another “liquorice stick” just like the one I grew up with. A Buffet Crampon E11 made in Germany. His passing reminded me that we all need a little jazz and big band in our lives. Grandpa danced to many a Benny Goodman songs in his life with a giant smile on his face. When I play and when I think of him, that is the memory I want to be present in my mind for him.

I will miss you Grandpa.


Withings Home Camera Review

I am always in search of products that help me automate and secure my home. I have run a mix of Foscam, D-Link, and Panasonics cameras dumping to a Linux server for the longest time, but it was clunky and not without its horrendous maintenance. Additionally the cameras, as aptly pointed out by Wife, were less than aesthetically pleasing.


Enter the Withings Home Camera setup. It checked off most of the boxes of requirements I had moving forward:

  • Doesn’t look like an ugly box with wires and lights when sitting on the shelf
  • Local streaming – I want to view the video via WiFi versus going to the internet to retrieve. I have a slow rural internet connection and video out is not an option
  • Ethernet – I don’t care for wi-fi if possible
  • Local storage – If I want to hook up a drive for local storage, I should be able to
  • Power Over Ethernet (POE) – This one unfortunately is getting very hard to find
  • Wide view lens – I hate having to mount multiple cameras just to cover a single room

Just for those that find this looking for reviews, it is a great setup. You can tell it is still a new product, but compared to the homegrown solutions you cannot go wrong. It is missing a few things such as IFTT, external API, Motion/Alarm times (if I am home I don’t need to know I am in the livingroom), and the ability to lock down ports for external access. The VOC air quality thing is nifty, but the propane heater in the house drives it batty. Two way comms for the audio is good when it works, but not very reliable on less than desirable WAN connections. WebRTC is the backend so I believe that is partly to blame as the comms channel matures.

From a security standpoint….

  •  I run a pretty tight firewall setup. I noticed external video access while away wasn’t working. A quick traffic dump showed the device was attempting to use ports 1024-65000 randomly. I emailed support and was told this is how it worked and to open all the ports. Nope. Fix it guys.


  • XMPP is used heavily for the messaging and transfers which is nifty. Everything is encrypted via SSL on a secure XMPP port. You can see from the dumps that the timeline feature is all exchanged via XMPP to this server: prod-oregon-timeline-2-days.s3.amazonaws.com
  • Each device is given a unique ID for sending the messages and retrieval. I haven’t tried spoofing or decoding, but here is the XML stored on the server:
<Message>Access Denied</Message>
  • Remotely checks into http://scalews.withings.net/ for status. I assume this is firmware or upgrades, but have not validated
  • When there is an alarm it uses a cgi-bin against the same scalews server for storing the alarm

Example Alarm