The Day the DNS Died / BIND Triage Server Array

(Credit Jon Watson)

DNS is such a pivotal, central, important part of not just internet browsing, but of everyone's IT infrastructure.... and yet, when it died, I was left to eulagize, alone, at its funeral.  I don't mean to be melodramatic, and I am not going to start singing any Don McLean songs, but I do want to complain.  I AM the IT department, so when something like this breaks and I come up with a great solution, I can't share with anyone.

The best I ever get to do is give analogies... and I usually send them before lunch so it usually has something to do with food or cars, or both:

Staff,

Our internet is running slow... Technically, our name resolution is running slow.  The internet is now running much faster.  It’s kind of like a pizza delivery guy who drives an Austin Martin DB9 with a super charged V8, but doesn’t know the area, doesn’t have a phone book and has a GPS device from the 90’s whose batteries have run out.  We have been having collisions on the network, and our DNS (internet phone book) is trying to resolve every one of them.  I am loading up a second DNS server right now, and separating internal and external traffic so we can get through testing, and you all can take advantage of our faster internet.  Please be patient this morning and I will keep you all up to date.  

- Tom

We should probably start at the beginning.  I am the sys admin for an "Urban-Ring" School in the midwest. We have a fairly standard setup. We have fiber running from every rack and between every building.  We have a standard AD Windows Domain setup with backup DCs. Our DHCP is also one of the Domain Controllers (DCs).  It feeds both the primary Windows DNS Server and the slave.  Because of the continual need for both internal and external resolution of names and addresses, our Windows machines serve up everything, and are the only DNS servers our clients see.  We have a SonicWall with content filter, firewall, etc... for a gateway.  All very standard.

I am very happy with our Windows servers.  They are easy to manage, they are scale-able, and they have always done exactly what I wanted them to.    We have several virtual networks on our one physical network.  We are in the process of running more fiber so that we have 2 more physical networks.  The problem entered in when we doubled the clients on the network.  We are in the middle of switching our security cameras to IP cameras, and we just added 55 IPads to the system.  We have the IP addresses, and we have the outgoing bandwidth.  But one afternoon, everything screeched to a halt.

I was frantic.  Running every test I know to run, I ruled out anything obvious.  In the end I found an overworked primary DNS server and a useless secondary, and lots of collisions on the network.  I needed a fast solution so that we could get through some standardized testing, and keep our IT services going until I could finish our expansion and put things right.

My solution was a BIND Triage Server Array.  You will not find anything on google about a BIND TSA, but is seemed like a good name for what I was doing.  Essentially, I needed a few forward only DNS servers to seperate out our internal traffic and outgoing traffic, and get it to the right place.   I wanted to continue using my DHCP and AD DCs for internal resolution because it is more efficient and verbose for Windows machines.  All of our IPads and such  were not going to be authenticated by the firewall and content filter at the sonic wall.  BIND is a fast, easy solution.

Here is how I did it:

Spin up CentOS 7, minimal install, headless.

Give the machine a static IP during install.  The DNS server is going to be changed latter, at this point use your standard DNS, or 8.8.8.8 so that it can download additional programs.

Once you are up, install BIND and its utilities, nano (text editor), wget (to update your root fowards), and tcdump (to monitor things).

$ sudo yum install bind bind-utils
$ sudo yum install nano
$ sudo yum install wget
$ sudo yum install tcpdump

OR the quick and clean method of installing them all:

$ sudo yum install -y bind bind-utils nano wget tcpdump

The first thing I do, before I start editing my config file, is make sure my root fowards are up to date.

$ sudo wget --user=ftp --password=ftp ftp://ftp.rs.internic.net/domain/db.cache -O /var/named/named.root

Now, we can edit.

$ sudo nano /etc/named.conf

I changed a few things to make it simple.  This is for a 192.168.0.0/24 test network, we use different subnetting, but that is for another day, 192.168.0.0/24 is simple.  Change the IPs to match your network.

The listen-on port is the internal and static IP of the DNS server.
Change your allow-query to your network
Set your forwarders as your ISPs DNS servers
Change the name of the root fowards file
The two other zones are for your internal network.  In this case 192.168.0.3 is the test networks AD DC, DHCP, and DNS server.

//
// named.conf
options {
        listen-on port 53 { 127.0.0.1; 192.168.0.2;};
        listen-on-v6 port 53 { ::1; };
        forwarders {
        8.8.8.8;
        8.8.4.4;
        };
        directory       "/var/named";
        dump-file       "/var/named/data/cache_dump.db";
        statistics-file "/var/named/data/named_stats.txt";
        memstatistics-file "/var/named/data/named_mem_stats.txt";
        allow-query     { localhost; 192.168.0.0/24;};
        recursion yes;
        dnssec-enable yes;
        dnssec-validation yes;
        dnssec-lookaside auto;
        managed-keys-directory "/var/named/dynamic";
        pid-file "/run/named/named.pid";
        session-keyfile "/run/named/session.key";
};
logging {
        channel default_debug {
                file "data/named.run";
                severity dynamic;
        };
};
zone "." IN {
        type hint;
        file "named.root";
};
zone "myschool.edu" {
  type forward;
    forward only;
    forwarders { 192.168.0.3; };
};
zone "0.168.192.in-addr.arpa" {
    type forward;
    forward only;
    forwarders { 192.168.0.3; };
};
include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";

Make sure you didn't jack it up....

$ sudo named-checkconf

Make sure clients can connect through your firewall.

$ sudo firewall-cmd --permanent --add-port=53/tcp
$ sudo firewall-cmd --permanent --add-port=53/udp
$ sudo firewall-cmd --reload

And, we want it to start on boot.

$ sudo systemctl enable named

and GO.....

$ sudo systemctl start named

Now we want to change our DNS server's DNS server, first check what the name of your NIC is.

$ sudo nmcli connection show

This DNS server's connection was p3p1.... so ...

$ sudo nmcli con mod p3p1 ipv4.dns "127.0.0.1"

At this point, change the DNS servers for your clients to the DNS servers static IP, and we can test the system.  The following command will let you log all the queries going to the new DNS server.

$ sudo rndc querylog

To actually see the logs....

$ sudo tail -f /var/log/messages

or get fancy with perl .... do install perl

$ sudo tail -f /var/log/messages | perl -pe 's/.*named.*/\e[1;31m$&\e[0m/g'

If you want more information, use tcpdump.  Note that you use the name of the NIC connection, in this case p3p1.

$ sudo tcpdump -n -s 1500 -i p3p1 port 53

Spin up a second one to make sure you are redundant.  I actually found that this works really well, and may leave it like this for a while.  It is easy, peasey, and quick to change.

Steaks and .22s

I have not had a nice wood grained .22 since I left college, so it was a treat to keep one in storage for a family member who was going to be spending a few years overseas.  This particular .22 rifle is a Marlin from the late 70s.  It spent almost 30 years in the humidity of Arkansas, being bumped around the woods of the Ozarks.  After that, it spent some time in a garage in Michigan.  It was a well taken care of gun, but it had suffered some from the weather, and lets face it, it had been a boy's first gun.

There was some checking in the wood, plenty of rust, and a slight warp in the stock.  The easiest thing would have been to strip everything down, straighten the stock, and re-finish... BUT, There is something to be said for preserving the years of use and the battle scars.  After talking to the owner I got to work.

2 screws removed the barrel/receiver assembly from the stock, were the stock was straightened and splits repaired.  After that the stock was cleaned and given a gentle re-finishing.  For a finish, I used a mixture of turpentine, boiled linseed oil, and apple cider vinegar.

Barrel was cleaned, treated, and re-blued.

Everything re-attached, and scope was sighted in.

BEFORE pictures are on the left, AFTER pictures are on the right.  Click on the pictures to enlarge.

Working on firearms, does put me in mind for grilled steak.  Grilling steaks have always scared me.  I have been grilling since I was 12, and I feel confident in burgers, brats, chicken, chops.  I'll smoke anything in the smoker... My steaks have always come out too chewy, or overcooked, or flavorless.  Last January, after eating at a great steak house, I resolved to solve my steak phobia.  The method I have ended up with is supported by various internet blogs, and Cook's Illustrated.  

Reverse Searing is really very simple, and makes great sense.  Essentially, after seasoning the steaks fairly simply... lots of salt, some pepper, a little garlic and onion powder... the steaks are put on the cool side of the grill.  On my grill, I turn on 2 burners and leave the steaks over the 2 burners that are left off.  I leave a remote temperature probe in one of the steaks and let it all slowly cook.  At 90 degrees F I flip them.  At 115 degrees F I put a little melted butter on both sides of the steak, crank up the hot side, and finish the steaks up to about 135 degrees on the hot side of the grill.  When the steaks come off, they each receive a nob of compound butter.  

The result is a nicely seared steak, with a nice pink/red band that is even all the way through from edge to edge.

The other part of my secret, is quality meat.  This is a Creekstone Farms Ribeye from Douglas Avenue Chop Shop.  When you watch for specials, you can get a great deal on awesome cuts of meat from a quality butcher. This particular steak was finished with compound butter made with Plugra butter and roasted bone marrow.

What do these have to do with each other, you may ask?  A quality grilled steak and a cleanly refurbished rifle are tributes to an attention to detail, quality workmanship, and quality raw material.  It is easy to get caught up in high priced steak houses and brand new rifles, but recovering a loved firearm or grilling a steak on your own equipment does give you a lot of pride and does, genuinely, add to the enjoyment of the experience over all.

End of Christmas Season Feast (Gluten Free Recipes: Cheesy Hash Browns, Green Bean Casserole, Spiced Cranberry Sauce)

Today, The Baptism of Jesus marks the end of the Christmas Season... so last night we feasted.  The tradition of having an extra Christmas feast... actually started with a faux Thanksgiving, years ago.  We had been visiting a friend shortly after Thanksgiving and we had decided to cook all the things that we really liked to eat, but couldn't convince the extended family to accept the "official meal."  Hannah and I love to cook, and we love to try new things.  In so many ways, going gluten-free was exciting because it gave us an excuse to try new things... or old things in new ways.  This is always one of our least healthy meals of the year... but it always tastes good

This weekend, Hannah's sister Sarah and her boyfriend were up visiting and were able to join us.

This year's spread included (going clockwise) Ham, Cranberry Sauce, Croissant Rolls, Loaded Mashed Potatoes, Peas, Cheesy Hash Browns, Green Bean Casserole, Sausage, Walnut, Spiced Cranberry Sauce, Gravy, Sweet Potatoes, and Corn.

Cheesy Hash Browns

2 lbs Shredded Potato (We shred our own with a box grater, but you can use the frozen)

8oz Sour Cream

2 cups Cheddar Cheese

1/2 Stick Butter

1 can cream of chicken soup (we use Pacific brand)

Salt, Pepper, Garlic Powder, Onion Powder to taste

Mix everything together, and top with crumbled cornflakes or potato chips.  Pop in the oven @ 350 for an hour.  We usually leave it for another 15 minutes so everything gets toasty.

Green Bean Casserole

Enough green beans to fill your container about halfway (we used about 20oz)

1 Can cream of Mushroom (we use Pacific brand)

1/2 Cup Milk

Mix everything together and bake @ 350 for about 30-45 minutes.  During the last 15 minutes, top with crumbled potato chips (or fry up some gluten-free onion rings)

Spiced Cranberry Sauce

1 package (12 ounces) fresh or frozen cranberries

1-3/4 cups sugar

1/2 cup water

1/2 teaspoon ground cinnamon

1/2 teaspoon ground allspice

1/8 teaspoon salt

1/8 teaspoon ground ginger

1/8 teaspoon ground cloves

Combine all ingredients.  Bring to a boil.  Simmer until everything thickens and the berries pop (30 minutes or so)  Cool.

Noodler's Konrad Rattlesnake and Adrenaline

I had been waiting for some time to try and snag one of the ripple ebonite Konrads from Noodler's.  I have always been impressed with Noodler's Ink.  I am not going to tote them as the perfect brand... because I am not sure one exists.  I have been using their ink for a long time, and I trust it.  I liked the original Nib Creeper pens, and really like the Ahabs.  I was a little disappointed in the original Konrad.  I did not think the fit and finish was there.

I really thought the ebonite Konrad would be a big improvement.  Like anything else, however, I am easily distracted when it comes to pens.  Then I saw a video Nathan (Noodler's CEO, Chief Chemist, etc) posted on you tube about the acrylic Konrads ... The John Mung Konrad.  

I am not a big fan of white pens... The picture below show most of the pens I have at the ready.  I have more in boxes and drawers, but these are the ones that are usually in active rotation.  I think I have a white Lamy somewhere... I am really a fan of blue or black... but the acrylic Konrads are really pretty.

I almost bought the John Mung pen.  The one I kept ogling, however, was this one... the Rattlesnake and Adrenaline acrylic Konrad.  

No, it does not come in a pretty sleeve .. it comes in the same box that all the Noodler's pens have.  I do think the white background really accentuates the chatoyance of the material.  It is somewhat transparent.  You can see a few threads about mid-way up the body of the pen.

The fit and finish is excellent.  There are no stray pieces of plastic, or metal fittings that slip.

I have only had this pen a few weeks, but I have used it quite a bit.  It is really a lovely pen.  I took more pictures than I normally do... simply because it is so hard to express in a photo how pretty it really is.

That is not the stock flex nib on the pen.  I have a Jinhao 159 turned into a desk pen that has a wet medium nib.  It has one of the wetter feeds of all my pens.  I use it at school on my desk to write notes on cheap paper, write discipline referral slips on REALLY cheap paper, and the like.  I wanted a finer nib... and after using the stock nib on this Konrad... I knew what I was going to do.  The desk pen now has a flex nib.  This is the 7th or 8th flex nib I have gotten from Noodler's and I will say it is superior.  I don't know what is so different about it, but it is easier to get it to flex and is far more consistent.

I used the opportunity of getting a new nib to try out Jowo nibs.  I usually get Bock nibs as replacements, but the Goulets have started selling branded Jowo nibs.  It is a nice nib, smooth and stiff... I prefer some give... but I have learned what to expect from modern steel nibs.  The jury is still out as to whether Jowo beats out Bock.  It is a nice nib.  I got a Broad... It fits the pen nicely.

The first inking was Waterman Florida Blue... then Noodler's Black... in the above picture it is filled with Pilot Iroshizuku Tsuki-yo.  At first I thought a colored ink would mess with the visual balance of the pen, but I find that I really like the bluish tinge that comes through.

If I were to have a complaint with the looks, it would be this little patch of air bubbles that settled in the acrylic.  You can see them in the picture above... just to the right of the ink window.  On the other hand... it almost looks like a batch of scales... it is called Rattlesnake and Adrenaline after all.

It is a comfortable pen... for me.  I thought the original Konrad was a bit small.  This one is comfortable.  I do not post my pens, but because the piston nob is under a blind cap, you can do so without risking turning the piston.

Compared to the other models of Noodler's I have, it is definitely one of the bigger ones.  From left to right: Neponset, Acrylic Konrad, Ahab, Original Konrad, Nib Creeper.

Compared to some of the better known pens, I think it can still be considered a larger pen.  From left to right: MontBlanc 149, Noodler's Neponset, Noodler's Konrad, TWSBI 540, Sheaffer Legacy II, Lamy 2000.

I will buy another acrylic Konrad when more colors come out.  If nothing else, these are a great price for a beautiful pen.  40$ is nothing compared to the money you could spend on an acrylic pen from one of the bigger companies, or to have one custom made.  I think they are very comparable to a pen from one of the major brands, but I think the fit and finish of a custom pen is still superior.

If you have been holding off from getting a Noodler's pen because you got frustrated with an early model Ahab, or have just been waiting for the Noodler's to put out a true "user grade" "works out of the box" pen...  this is your pen.  This is not just a pen for tinkerers... etc... this is a solid, beautiful, functional fountain pen.

UPDATE:
I realized that I mentioned the new Jowo nib... here is how broad the broad is:

Ink Journal Scans Part 1

I have been trying to keep an ink journal for a few months now.  I am now finding that the hardest thing is now making scans that are true to the color of the ink.  The paper is pure white Clairefontaine 90g/sm paper.  The first page has almost too much contrast, and the others, maybe too little, but it will get better as I go along.  The first date is when it was inked, then the ink, then the pen, with nib size, and finally there is a date in parenthesis when I flushed that ink out of that pen.  

(Noodler's Liberty's Elysium, Noodler's Kung Te Cheng, Pharmacist Urkundentinte, MontBlanc Oyster Grey, Pilot Iroshizuku Yama Guri, Waterman Serenity Blue, Iroshizuku Kon Peki, Noodler's Bulletproof Black, Noodler's Green Marine, Noodler's Black Swan in English Roses)

(Diamine Poppy Red, Pharmacist Turkish Night, Iroshizuku Kon Peki, Noodler's Bulletproof Black, Noodler's Kung Te Cheng, Waterman Serenity Blue, Pharmacist Turkish Night Diluted, Noodler's Bulletproof Black, MontBlanc Irish Green, Pharmacist Urkundentinte, MontBlanc Oyster Grey)

(MontBlanc Toffee Brown, Noodler's Bulletproof Black, Iroshizuku Tsuki yo, Noodler's Apache Sunset, Waterman Serenity Blue, Noodler's Polar Black, Noodler's Liberty's Elysium, Waterman Serenity Blue, MontBlanc Irish Green,  Noodler's Black Swan in English Roses, Iroshizuku Kon Peki)

Sheaffer Legacy 2

I know that I should never walk into a brick and mortar pen shop without an idea of what I am coming to look at... 

Over the Easter weekend (end of March, beginning of April, 2013) my family was in Little Rock.  I have a habit of visiting several places... not least of all, Vanness Pens.  As much as I would like to make a plug for them, the reality is, they still exist because they do what they do so well, and their work speaks for itself.   

Needless to say, there are a handful of pens that I have bought online, but I generally try out the more expensive ones... you never know... there are always little differences even within the same make and model.

I went in with some intention of test driving a sailor pen, trying to talk myself into a Parker Vanishing Point, pondering the Faber-Castell basic, I even thought about the Waterman Exception.  I loved the nibs on the Sailors, not sure about the body... still can't get over the clip placement on the Vanishing Point... the Basic did not sit right with me... and I am still fascinated by the Waterman Exception.

I perused the cases... tried several pens... 

Eventually, I made it to the Sheaffer case.  

I had made an offhand comment to Mike Vanness several months ago about my perceived decline in the quality of Sheaffer products, he told me I shouldn't give up too easily on them, they still had several great products.  Several months ago I tried a Legacy and I wasn't sure.  I liked how it felt, I liked the nib... but I couldn't get over the "modern Sheaffer" hang up.

Shortly after I had tried the Legacy... I acquired a MontBlanc 149.  I did a writeup here... Review of the MontBlanc 149 by a former anti-montblancite

Somewhere in there FPN's jar reminded me of the Legacy and of PFMs... That has stayed on my mind...

So THIS trip, after trying it once before, after writing with a MB 149 for several months... when Mike took the Legacy 2 out of the case... it didn't take much convincing.  I haven't looked back.

I got the black lacquer with gold trim.  I have wondered if I should have gotten one with a silver cap... but when I see pictures of them all together, my eyes are always drawn to this one.

 It is a hefty pen.  Lite pens make my hand cramp.  It is not in any way unbalanced.  I am not in the habit of posting my caps, but I could if I wanted to, and it would not make the pen unwieldy.

I love the inlaid nib.  I went for a medium, and I would say it is fairly true to that.  The section is also very nice.  I would say it has a very similar girth to a Lamy 2000, but it barely tapers.  The cap is held in place by an O-ring on this model.  I have tried several hand holds, and they are all comfortable.  I wrote lesson plans and lecture notes with it and did not have any problems with fingers slipping on the lacquer or finding any annoying bumps, threads, etc.

In my ignorance, I had glossed over the filling mechanism until after I bought it.  I know the pen is a reincarnation of the PFM (Pen for Men) and I am aware of the touchdown filling system... but it really is a treat.  It really works well.  I also like the idea that I can swap out the touchdown adapter with a cartridge.  I don't think I have used a cartridge in close to 15 years... but the adaptability is nice.

I will admit to being mystified by the touchdown filling system.  I understand that it compresses the chamber when you depress the plunger... essentially squeezing the sack inside the converter, and that a small grove in the end of the rod allows air back in, allowing the sack to return to normal and suck in ink... but how does it let air in when you are pulling the plunger out... The only thing I can figure is that the plunger rod tapers ever so slightly at the end...

Also, how difficult are these to repair if something breaks?  I guess there is always the cartridge or squeeze converter options...

Comparing it to other pens... from left to right: Noodler's Nib Creeper, Sheaffer 330, Lamy 2000, Sheaffer Legacy 2, Noodler's Ahab, Lamy Safari, MontBlanc 149.  I thought about throwing a few others in there... but I had a black pen theme going... so I stuck with it.

I am happy with the pen.  If you have been on the fence and can find one... go for it.  I don't know about the Legacy 1 or the Heritage... but I like the Legacy 2...

Jinhao 159 Desk Pen

I have been in the market for a desk pen.  I have looked at several of the ones available... and have come up short.  I find the ultra skinny pen... painful.  I have looked at the MontBlanc 149 desk pens, but I am a school teacher, and the price is prohibitive.

So, my solution was to make my own.  I would not recommend this with any pen you have sentimental value for... or that can't be replaced for less then 10$.  If you read the title and thought Jinhao had come out with a desk pen... I am sorry to disappoint.

We start with a piece of beveled wood.  I decided I like it a little square, but round works well.  If you don't have a router, it is easy to find pieces this size at hobby and craft stores.  It was important to me to have a little heft... but probably not necessary.  Part of the reason I went with the Jinhao 159 is the amount of metal in the pen.  I don't think it is going to put undue stress on the pen or cap.

I removed the cap finial and the clip   The finial is brass.  I don't think this would have worked well if it was not metal.  I carefully drilled a small hole and inserted a screw.

I drilled a larger hole in the wood, so that I could angle the pen down when it was drilled in.

I dry fitted everything to make sure it was going to work.

I stained it with some cherry wood stain... I liked it but realized I wanted more shine.

I am very happy with it.  It is sturdy, it is easy to remove the pen and I am happy.  

I really like the actual pen a lot.  I have heard people complain about the lacquer chipping off, etc.  I like to think this will help avoid that.