My website is down. Now what? Part 3 – Routing

This is part of a series on diagnosing your website outage issues. This is part three; links to the other parts are here.

In Part 1 of this series we covered the overview of what could have broken to cause your website to go down. In Part 2, we started working through those possible issues by diagnosing DNS issues. Now that we know your domain’s DNS is good, we’re going to start looking at the routing. In other words, now that we know where your domain is (DNS), are there any roads to get there (routing)?

Many people misunderstand how the web works. I blame the media and well-meaning explainers for suggesting and continuing to propagate the notion that we “log on” or “visit” websites. The problem with these terms is that they naturally call to mind the concept of “going” to a website which is the exact opposite of how web surfing works. Instead of your browser being some sort of vehicle that goes to websites, it is a static piece of software that sits on your computer and issues demands for websites to come to it. Websites come to you, you do not go to them.

Internet routing imageWhen you click a link or type in a domain name, your browser issues a request across the Internet to that domain’s web server and says “give me that page”. Well behaved web hosts comply and the content your browser requested is downloaded to your computer and your web browser then renders it nicely so you can see and interact with it. In order for this to happen, there has to be a pathway on the Internet that can transfer your web browser’s request to the web server and transfer the response (content) from the web server back to your web browser. Those pathways are called routes and a big part of website troubleshooting involves diagnosing routing issues.

Continue reading “My website is down. Now what? Part 3 – Routing”

Series: My website is down. Now what?

Panic button imageThis is a multi-part series about how to keep your cool when your website is down and be part of the solution instead of a seething mass of non-helpfulness.

This tutorial assumes the reader has some basic knowledge of how websites function as I believe it is unforgivable to be wholly and wilfully ignorant of the workings of things we buy. However, I maintain a quick primer here on the various moving parts of websites and their ecosystem should you need a quick refresher.

  1. Part 1: Overview
  2. Part 2: DNS Issues
  3. Part 3: Routing
  4. Part 4: Layers (firewalls, etc.)
  5. Part 5: SSL/TLS and HTTPS problems.

My website is down. Now what? Part 2 – DNS Issues

This is part of a series on diagnosing your website outage issues. This is part two; links to the other parts are here.

In Part 1 of this series I covered the fact that your web browser needs to know the IP address of your website which is done via a process called domain name resolution. This happens under the covers and is facilitated by your domain’s name servers which are part of the Domain Name System. This domain name to IP address resolution is absolutely fundamental to the functioning of the web and if there are issues with your domain’s DNS records, or your local computer’s DNS, you won’t be visiting your website today.

How to identify a DNS problem

Computers have local DNS caches and even networks sometimes cache DNS records. When those caches exist, your computer’s query for an IP address will never hit your actual name servers. That means if those caches are wrong, your computer will have no idea how to access your site. So, how do you test this out?

The first piece of information you need is the IP address that your website is supposed to point to. For most of us, that is the IP address your web hosting service provided to you when you signed up. If you have other layers (as described in Part 1 of this series) then the IP address would have been provided to you by that service provider. If you’ve lost that information, then you can open a support ticket with that provider and ask them what IP address your domain should be pointed to. Armed with that information, you can then compare that to the IP address your computer thinks your domain is at.

The easiest way to check what IP address your computer associates with your domain is to run a simple ping command. You can do this from the command prompt of any Windows, Linux or Mac computer like so:

Your output may look slightly different but the first line-ish will show you the IP address that your computer thinks your domain is at. In my case, this is 192.124.249.6 which is correct. I know that my DNS records are supposed to point to that IP address as that is my firewall IP address.

If my output contained some other IP address like so:

That’s a big red flag.

Continue reading “My website is down. Now what? Part 2 – DNS Issues”

My website is down. Now what? Part 1 – Overview

This is part of a series on diagnosing your website outage issues. This is part one; links to the other parts are here.

Website down! Aoooogah! Aoooogah!

ERR_NAME_NOT_RESOLVED
Google Chrome showing a DNS error “ERR_NAME_NOT_RESOLVED”
We’ve all had that feeling – you go to your website and it’s not there. It’s down. There may or may not be an error message giving you some clue what happened and now you’ve got to figure out what to do.

First thing’s first – focus on figuring out where the break is. Don’t revert immediately into victim mode and start opening angry support tickets at every person who has ever touched your website in an all-consuming witch hunt. It’s your site, take control.

Understand your architecture

Everyone who owns a website has their stuff set up differently. Domains are purchased from somewhere, sometimes years ago; a web hosting service hosts your website somewhere; some company handles your name servers (Domain Name Service) and you may have other steps along the way such as a Content Distribution Network (CDN) or firewall. Only you know all this stuff, so bring that information to the table.

Understand the problem

“Down” is not a useful description of a website’s condition. If your website is not displaying, it’s useful to be more specific.

Does your web browser just show a blank white page with nothing on it? Is there an error message somewhere on the page that explains what has happened? Does your website load partially, but have some missing elements? Define what “down” means because the particular symptoms you’re seeing will lead to better investigations into the root cause.

Continue reading “My website is down. Now what? Part 1 – Overview”

OpenSSH v7 and DSA key support. AKA “Permission denied (publickey)”

sshI have a little personal server farm with a handful of hosts that run things like my websites, a BBS and my VPN server. I recently upgraded my desktop to Kubuntu 16.04 and suddenly my SSH key was no longer working. I started seeing this when I tried to log in:

I began troubleshooting to determine what was wrong with my key. I had just upgraded my workstation, after all, I could have restored the wrong keys from my backup. That is when I discovered that I was able to log in to some of my servers, just not all of them. That puzzled me because I knew I had not touched the servers and I thought that they all used the same key. What could cause my key to work on some servers, but not others? It had to be a client-side issue but I didn’t know what.

I have a continuity plan to access my servers if anything like this should happen so I implemented it and it allowed me to look in the auth.log. I saw these messages in the server logs:

Continue reading “OpenSSH v7 and DSA key support. AKA “Permission denied (publickey)””

Defeating keyless entry front door locks.

I’m the least mathey person I know. My bio will attest to that – my skills are terrible but my curiousity is high. There’s a certain magic to numbers that I get a glimpse of every now again when I manage to win a struggle with them and it’s compelling to me. Math is a representation of data and while me and Math don’t along very well, me and Data are best bros. I spend my days mucking about in log files on other people’s systems looking for reasons, root causes, and footprints. The trails become clear once you tame the data and turn thousands of unruly log lines into succint sorted output. These same techniques are used by good guys and bad guys alike and from them we learn that some things are truly hard. We also learn that some things only look hard, but really aren’t.

Four digit numbers crop up repeatedly in our society. In the late 1990’s I had a TD bank account and my bank card had a 6-digit PIN. That did not last long because the international consortium of bankey people standardized on 4-digits for PINS which is too bad because that exponentially decreased the security of my PIN. Overnight the odds of guessing my PIN plummeted from 1 in 1,000,000 to 1 in 10,000. But, hey, the bankey heads know what they’re doing, right? But I digress…

I’m not sure how we landed on 4 digits, but that frequency turns up all over the place. My bank card PIN is 4-digits, my credit card PINs are 4 digits, even my front door lock is 4 digits. That begs the question: how long would it take to guess the code to open my front door? Let’s ask math.

Continue reading “Defeating keyless entry front door locks.”

Jon Watson – PGP Key

For email to me@jonwatson.ca

The fruit decision: how low should your website hang?

Probably the most significant decision people will make when building their website is the decision about what software to use. A lot of people choose existing CMS or ecommerce apps like WordPress or Magento which makes for a quick setup and reasonable support. Others choose to build their site from scratch or use one of many lesser deployed apps like the Ghost blogging platform or x-commerce. It’s nice to think that everyone evaluates the features of each offering and chooses the one that best fits their needs, but that is not what happens.

Most websites are owned by non-technical people without IT support so the software they end up using is whatever has the lowest cost of entry. That means whatever is in their control panel that can be installed with one click is what gets used.

Please use the back door sign

This situation is what leads to lopsided software deployment statistics such as massive WordPress footprints and, to step away from the Internet for a second, the global market share of Windows. The web software that is best at getting into one-click installers like Scriptaculous or pre-installed on desktop computers become the most popular. These large deployments of identical software provide a good selection of attack vectors for bad actors. If a vulnerability is exposed in WordPress, for example, a bad guy has literally millions upon millions of WordPress websites to attack using that exploit.

Continue reading “The fruit decision: how low should your website hang?”

A primer on your phone, I mean…your website.

The world of things is grouped into three categories for me. There’s things I know, things I don’t know, and things that in order for me to understand they even exist you’d have to go back to the Big Bang to give me enough context to get a grip on. I think that most of us think that most people know what we know, or at least have enough context to get up to speed pretty quickly. Recently, however, I find myself talking to a lot of end-user website owners and I’ve come to realize that is not so. I’ve had to have many Big Bang conversations with website owners in order to explain what I felt were pretty fundamental pieces of the Internet. So, I thought I’d try to lay out the basic things that I think everyone that owns a website needs to know.

Confused

Many of the people that cross my path daily are legitimately trying to understand all the moving parts of their website; but there is a sub-community that promotes willful ignorance as well. In some circles it has become chic to be incompetent with technology. We wouldn’t dream of saying things like “I take my car to work but I have no bloody idea how to drive” or “Lawnmower? Not a clue how it works, when it runs out of gas I just throw it out and buy a new one because I have no idea where the gas goes in”. But it is somehow OK, and in fact fashionable, to say “my website? Not a clue how it works. When it stops working I just scream and yell at random people until someone fixes it”.

So here’s my attempt to help.

Continue reading “A primer on your phone, I mean…your website.”

One man’s trash is another man’s treasure, yo.

Of all the things that interest me most in the world, I think humans might top the list. I’m a scientist at heart and I love to categorize things and learn the rules that apply to situations. Humans have this manifest ability to evade any and all attempts to be classified which simultaneously fascinates and repels me. The cultural concept of value is a good example. Value is one of those human things that is entirely subjective and just can’t be nicely typed. One man’s trash is another man’s treasure, yo (Peter Griffin would have loved this). But for certain values of trash, it’s treasure for all. Continue reading “One man’s trash is another man’s treasure, yo.”