Using Airwall: Secure Remote Access

Using Airwall: Secure Remote Access

In this blog post I would like to share the configuration I have been trying out with Airwall from Tempered Networks over the past months. Airwall is a new approach to security leveraging the HIP protocol defined in RFC7401. 

If you would like to read more about my experiences with Airwall, check out my first posts, Tampering Visibility with by Tempered and Configuring Airwall.

 

Network topology

The following topology describes how I have included the Airwall gateway in my home network. 

In this setup, I created a seperate VLAN for the untrusted VLAN that is effectively a subinterface on my FTD device. It allows traffic to the outside only. I have configured PAT to allow the necessary inbound Airwall communication ports. 

The trusted port of the Airwall gateway is directly connected to the client VLAN in my network. Why? I wanted to test out bonjour and L2 extension / forwarding too. 

Secure remote access

Probably one of the most visible use cases for Airwall is allowing secure access as if it was a VPN/SASE client allowing your users connecting securely to inside resources. And that is definitely possible with Airwall. You can leverage the gateway function to effectively connect secure (inside) hosts and networks via the secure ports.

For this use case, I want to securely allow access from my laptop to several management IP’s in my internal network, being:

  • Firepower Management Center (FMC)
  • C9800 Wireless Controller
  • My Raspberry PI with Domoticz to see real-time power consumption
  • My laser printer
  • My distribution switch (SSH)

As the gateway has its secure port to my clients VLAN, I just need to define these devices (and yes, IP scanning is possible) as local devices and add static routing for my network management. All this is configured for the specific Airwall via the Conductor (Airwalls -> Select the right Airwall). 

Add local devices to an Airwall gateway

Configure Overlay IP routing via the gateway.

I had some connectivity issues when I had my Mac conencted to the same VLAN and was not able to connect to my FMC. This is solvable by either setting a different overlay IP address on my Airwall Agents or configure PAT on the gateway. I’ve chosen the first option, and it has worked quite well. 

Now that the Airwall components have the right (local) connectivity, it is time to configure a security policy for secure access. Within Airwalls paradigm, these security policies are overlay networks. And they work very easy. Within the conductor the tab “Overlays” is where you’ll configure all overlay networks. I have created a hub-spoke overlay network with my own devices (which I grouped) as hub. Why would I do that? It means that all the other devices are spokes and I can automatically communicate with them, once I add them. 

Adding/Removing devices is like really easy. Just hit the plus and add any device you have defined in the Conductor, and that’s it. 

Once I add a device in the overlay, it can communicate with the hub. And when I am outside, or even when I’m at home, I am actually connecting to these devices via the Airwall secure network, as can be seen via the below traceroute.

In summary, I have been very satisfied with the way Airwall is working. This common use case is really easy to configure and manage. And as long as the underlay ports can communicate, it is fast and reliable. But you can do more with Airwall. One of the things from Security Field Day that got me triggered was the full programmability, which I will describe in another post. 

Configuring Airwall

Configuring Airwall

In a previous post I have already shared some information about the Airwall solution from Tempered Networks. In this blog, I will share my experiences in configuring and operating the Airwall solution. I have been running an Airwall setup to test for a number of months.

Airwall components

Before moving into the configuration aspect, let’s first describe the components of the Airwall solution.

Airwall Conductor

The conductor is the “heart” of the Airwall solution. It is the SDN controller (running on-prem or in a cloud) with which you manage all Airwall components and configure the different policies.

Airwall Agent

The airwall agent is a software component that is running on your user endpoint devices, such as your smartphone, tablet, laptop or desktop. It is effectively the client that connects to gateways and Airwall Server agents. 

Airwall Server agent

The Airwall Server agent is the software component that you can install on individual servers. You might say that it is comparable with the Airwall agent, except it runs faceless and is optimized for running on your servers. I have been running one on a virtual server I have hosted in the cloud. 

Airwall Gateway

The airwall gateway is a special component within the solution. You can use it to secure networks behind a gateway with agents. E.g., if you allow a connection from an agent to a gateway, you are able to securely connect from the agent through the gateway to multiple hosts or networks behind the gateway. The gateway comes in different flavors, a virtual machine, a cloud instance, a physical gateway for wired connections, and a phyisical gateway that connects to the unsecure networks via WiFi. The gateway is also capable of acting as a relay agent between agents that cannot communicate directly with each other, but only if you have specified and allow that communication in the conductor.  

Overlay networks

Overlay networks within Airwall are a bit different compared to the traditional overlay networks seen with VPN solutions, which is also what makes Airwall unique in its approach. An overlay network is basically a secured IP network that can be configured between agents, servers, and gateways. And the unique twist is that you can manage completely different overlay networks and configure a policy that a single agent can communicate via these different overlay networks.

My personal experience is that I initially misinterpreted overlay networks and thought of it as a single network with unique policies; it is easier to see each overlay network as a security policy that defines who can securely communicate with who at which moment in time. 

Underlay network

The underlay networks is your regular network, or the Internet. It is the insecure network via which you connect the different Airwall components. The conductor of course needs to be reachable via the underlay network.

Getting started

The team at Tempered networks was kind enough to provide me with a complete set of components to test with and try out use cases. My kit consisted of agents for my iOS devices, my laptops, 1 virtual gateway (Cloud), 2 physical gateways (one wired, one wireless), a server agent and a conductor.

Getting started with the conductor is pretty much straight forward, after running through the provisioning guide, you have the link to the conductor portal, which provides a good overview of your environment. The tab labels itself speak for itself. Now that the conductor is setup correctly, it is time to download and provision agents.

 

For the physical gateway, it was effectively the same setup. Hook a serial console to the device, configure the hostname and port for the conductor, activate the device and you can define your policies.

I used the provided download links to download an agent to my Mac and used the App-Store to download the iOS apps. Once they are downloaded, you provide the agent with a profile that contains the hostname and port to your conductor. It took me a while to figure out that I had to manually approve the license before further configuration was possible. So after you’ve configured your agent, go back to the conductor and hit the settings -> Licensing tab. Activate the just registered device and a license will be consumed. With that action, the airwall agent/gateway becomes active.

Remember, the gateway has different behavior compared to the agent with more functionality. Make sure that the communication ports you defined in the Conductor are available via the underlay networks. It took me a while to figure that out (I assumed too much routing in my mind), and once I had the PAT configured correctly, the gateway was working as expected and the agent on my mac could communicate with it. If you use NAT, you can configure the external IP address for the gateway via the conductor (auto-detect and auto-set would be great when you have dynamic outbound IP-addresses and relay on PAT). The screen shot below shows the setting for my gateway.

In my next post, I will give you a sample underlay network that I have been using very succesfull (including demos and firepower trainings) over the past months.

Configuring Airwall

Finding out which MIBs are supported on your Cisco device

Recently fellow champion Ioannis Theodoridis asked around how to find out what SNMP MIB’s are supported by a specific Cisco network device. And although model driven telemetry is gaining momentum, many monitoring systems still rely on SNMP. And finding those MIB’s can be a challenge. I found out a very easy method to quickly find which MIBs are supported. 

To get started, you need your CCO credentials by hand and an outbound FTP client. In this blog I’m using the terminal app on my mac.

1. Connect via FTP to ftp.cisco.com and login with your own CCO credentials, set FTP to passive

pr-mbp15-001:Desktop nefkensp$ ftp ftp.cisco.com
Connected to ftp.cisco.com.
220-
220-	Cisco Systems File Transfer Service.
--- SNIP ---
220 download-prod1-03.cisco.com FTP Server (Apache/2.2) ready.
Name (ftp.cisco.com:nefkensp): nefkensp
331 Password required for nefkensp
Password: 
230 User nefkensp logged in
ftp> passive
Passive mode on.
ftp> 

2. now go to the directory “/pub/mibs/supportlists”

ftp> cd /pub/mibs/supportlists
250 CWD command successful.
ftp> 

3. Do a directory listing (long) and enter the directory for your device. The device listing is long, I selected a new Cat9k switch in this example 

ftp> ls
200 PORT: Command successful
150 Opening ASCII mode data connection for file list
drwxrwxr-x    2 swdsadm  cisco        4096 Jan  7  2002 2948g-13
drwxrwxr-x    2 swdsadm  cisco        4096 Jan  8  2002 4908g-13
drwxrwxr-x    2 swdsadm  cisco        4096 Dec 20  2000 90i
drwxrwxr-x    2 swdsadm  cisco        4096 Apr 19  2011 CTXSystem
drwxrwxr-x    2 swdsadm  cisco        4096 Jan  6  2003 ONS15530
drwxrwxr-x    3 swdsadm  cisco        4096 Dec 20  2000 accessProEC
drwxrwxr-x    3 swdsadm  cisco        4096 Dec 20  2000 accessProRC
drwxrwxr-x    2 swdsadm  cisco        4096 Aug 13  2008 ace
--- SNIP ---
drwxrwxr-x    3 swdsadm  cisco        4096 Oct 27  2006 wsc6009
drwxrwxr-x    2 swdsadm  cisco        4096 Oct 27  2006 wsc6503
drwxrwxr-x    2 swdsadm  cisco        4096 Oct 27  2006 wsc6504
drwxrwxr-x    3 swdsadm  cisco        4096 Oct 27  2006 wsc6506
drwxrwxr-x    3 swdsadm  cisco        4096 Oct 27  2006 wsc6509
drwxrwxr-x    2 swdsadm  cisco        4096 Oct 27  2006 wsc6513
drwxrwxr-x    2 swdsadm  cisco        4096 Dec 20  2000 wsc8510csr
drwxrwxr-x    2 swdsadm  cisco        4096 Apr 11  2001 wsc8510msr
drwxrwxr-x    2 swdsadm  cisco        4096 Dec 20  2000 wsc8540csr
drwxrwxr-x    2 swdsadm  cisco        4096 Apr 11  2001 wsc8540msr
drwxrwxr-x    2 swdsadm  cisco        4096 Jan  9  2008 xr12000
226 Transfer complete.
ftp> cd cat9300
250 CWD command successful.
ftp>

4. Do a directory listing and fetch that HTML file and logout

ftp> ls
227 Entering Passive Mode (72,163,7,54,102,29)
150 Opening ASCII mode data connection for file list
-rw-r--r--    1 swdsadm  cisco       15983 Jan 29  2018 CAT9300.html
226 Transfer complete.
ftp> get CAT9300.html
227 Entering Passive Mode (72,163,7,54,102,32)
150 Opening ASCII mode data connection for CAT9300.html
226 Transfer complete.
16159 bytes received in 0.169 seconds (93.1 kbytes/s)
ftp> 

5. Open the HTML file in your favorite browser and voila, you have the supported MIB directory for that network device type

Below are two samples of the supported MIBs for the Cat9300 family and the Nexus 3000 family. 

Summary

I have found this method very easy to find which MIBS are supported by any cisco device. It is an easy way to get that comprehensive list of supported MIBs. If you want to download them for your own NMS, you can go to the SNMP Object navigator and download the supported MIBS as next step.

Your content goes here. Edit or remove this text inline or in the module Content settings. You can also style every aspect of this content in the module Design settings and even apply custom CSS to this text in the module Advanced settings.

Configuring Airwall

Cisco C9800-CL sits idle at GRUB Loading Stage2…

I have been using the Cisco Catalyst 9800-CL (Wireless Controller for cloud) for a while now. Recently, I accidentally powered off the wrong VMWare server, resulting in a wireless disruption. Priority 1 at home! And of course, just before I had a session with Shawn preparing for CiscoLive Barcelona…

After restarting the vSphere server, my C9800-CL wasn’t booting up, with a message: “GRUB Loading stage2… ” And it just sat there, for minutes..  Eventually, during the WebEx Call, I managed to fix it and got my controller back up and running. 

Steps to fix the issue

These are the steps that I used for fixing this issue.

First, power off the VM in vSphere. We need to change some settings in the BIOS.

Next, go and select “Edit Settings” of your VM and click “VM Options” at the top to view some advanced settings and click “Boot options” open. Change the Boot delay to “8000” milliseconds, so that you have enough time when you boot the VM.

Hit Save after you have changed the settings.

Just to make sure, open the settings of the VM again, and click open the first CD/DVD Drive. Check that there is an image named “_deviceImage-0.iso” and that it is connected at powerup.

When I used the vCenter convertor to move the VM off to a new server, I found that this iso wasn’t copied with the controller and it is needed.

Hit Save when you know the ISO image is there.

Follow the next steps to get the C9800-CL booting up again

  1. Open up the console of the VM in the browser (it saves you time)
  2. Power on the VM
  3. Once the Bios is shown, click in the console and hit “ESC
  4. The boot order menu is shown, like the image on the left
  5. Scroll down to highlight “CD-ROM Drive” 
  6. And hit “Enter
  7. Now the VM will boot normally and your controller will start as expected.

Summary

It seems that Grub (the bootloader on the first disk) is not configured correctly the C9800-CL, which leads to a VM / Appliance that is not booted because it cannot find any kernel to load. By selecting the CD image, the right bootloader is selected and the controller is started with the correct configuration. 

I do assume this is a caveat/bug in the Cloud version and will be fixed in a newer release. I do hope you can use this info to fix your C9800-CL deployment sooner. 

FDM Application fails after upgrade

FDM Application fails after upgrade

This is just a quick blog post for those that might have FDM issues after upgrading your FTD software.

I have recently updated my Firepower appliance from 6.5.0 to 6.5.0.2. One of the reasons to update is not only that 6.5.0 is a .0 release, but also that I noticed some failed rule-update deployments that set snort to block all traffic.

Unfortunately, after upgrading, FDM reported an error that it could not be launched with an application failure error. The suggested action was to remove the manager, add a new local manager and begin from scratch. This is the error: “The Firepower Device Manager application cannot be opened. Please try again”

While googling for a possible caveat of this behavior on 6.5.0.2, I came across a caveat in 6.2.3 that has the same behavior. 

That caveat has supported me in fixing my solution. What I did was executing the following commands:

 

> expert
**************************************************************
NOTICE - Shell access will be deprecated in future releases
         and will be replaced with a separate expert mode CLI.
**************************************************************
admin@na-grm-ftd01:~$ sudo su -
Password: 
root@my-ftd01:httpd# cd /ngfw/var/cisco/ngfwWebUi/
root@my-ftd01:ngfwWebUi# ls -a
.   .bootstrap-failed  clifile    deploy                      ha_pkg  lina_cli_sqlite_stores   pjb_output  sslCiphers  variables.ftd_onbox
..  bin                clisyncer  ftd_onbox_6.5.0.2_previous  libs    ngfw_onbox_bootstrap.sh  sru         tomcat      version

root@my-ftd01:ngfwWebUi# rm .bootstrap-failed 
root@my-ftd01:pmtool disablebyid tomcat
root@my-ftd01:pmtool enablebyid tomcat

Basically, you go into expert mode, find the tomcat directory used for FDM and then remove a status file and try to restart it.

With me, this worked and helped me get back access to FDM. Should you run into issues with FDM after an upgrade, this “hack” might help you.

Disclaimer: You are entering expert mode of FTD, it means you can DESTROY your FTD configuration and box. Be aware of what you are doing and make sure you have a backup. 

Swift, JSON Encoding/Decoding and subclasses

Swift, JSON Encoding/Decoding and subclasses

Over the past weeks I have been preparing for two CiscoLive Barcelona breakout sessions. In one of them I will give a brief demo and the other session where I will be covering parts of the Cisco Press book that I wrote. The preparation itself is not only about the slides, but also developing code that is to be used in the demo’s. These demo’s are built on iOS devices and run on some containers, so I have been writing that software in Swift, which is a beautiful and powerful programming language. One of my previous posts covers some principles of Swift. One really powerful feature is the easy capability to encode or decode data to the JSON format.  

If you want to have a class to be able to convert to and from a JSON format, just use the Codable protocol and you’re ready, see the code example below:

/*
 * Enumeration of supported message types. Extend this for new messages
 */
enum MessageType : Int, Codable {
    case unknown = 0            // default, unknown
    case acknowledgement = 255  // acknowledgement to message, if required
    case hello = 1              // hello, for keep alive, always followed by ack
    case sendMessage = 2        // send a unicast message to another client
    case broadcast = 3          // send a message to all connected clients
}

/*
 * Generic parent class
 * Every message has the following attributes
 * Version: To define which version we are talking about
 * Command of the message
 * client-id that sends the message
 */ unique request id, used for acknowledging, etc..
class Message : Codable, prettyPrint  {
    var version : String = "1.0"
    var msgType : MessageType = .unknown
    var clientId: String = "" // client host, generated by the server to guarantee
    var requestId: String = UUID.init().uuidString  // unique request id for this message, used in the ack
    
    // Default constructor
    // Not used cause calling super.init can override msgType value
    init() {
        // empty on purpose
    }
}
This code example defines a class message with variables for messageType (of type MessageType), requestId, which is a unique UUID string value, and a data variable which can contain any String. So let’s say I create a new message , called hello with the data “Hello there!” with the following code sample:
let msg = Message()
msg.msgType = .hello
msg.data = "Hello There!"

To convert this to JSON, this would only require a few lines of code:

let encoder = JSONEncoder()
let jsonData = try encoder.encode(msg)

The variable jsonData (of type Data) now contains a JSON-version of the earlier created message. Just to check the output, I can use the following commands to convert that data to String and output it in XCode’s Playground. 

let jsonDataAsString = String(data: jsonData, encoding: .utf8)

Suppose you would like to extend our message class with a special broadcast message, where the message can be sent to a all endpoints.. You could add an optional broadcastContent variable to the message class and create a state machine to determine when to use that value. Another alternative is to leverage the power of object-oriented programming and create a new subtype, like the following code example:

/*
 * BroadcastMessage is used to broadcast a message to all connected clients
 */
class BroadcastMessage : Message {
    // response message
    var msgContent : String = ""   // Message to broadcast   
}

So when you’d create a multicast message, like below, you’d expect that it would contain all attributes in the json file, right? Let’s check it out in Playground:

As you can see, the output does not contain all attributes of the broadcast message! It only contains the base message type class values. The msgContent variable is not included. It took me some time debugging and researching to figure out what happens. Swift bug SR-5431 and SR-4722  provide more details. Without going into those bugs, it comes down to the fact that as soon as you subclass a class that conforms to Codable, you need to override the default encode/decode methods and write your own. After some fiddling around, I have used the following code pattern to achieve that result.

/* Generic parent class
 * Every message has the following attributes
 * Version: To define which version we are talking about
 * Command of the message
 * client-id that sends the message
 * unique request id, used for acknowledging, etc..
 */
class Message : Codable, prettyPrint  {
    var version : String = "1.0"
    var msgType : MessageType = .unknown
    var data: String = "" // client host, generated by the server to guarantee
    var requestId: String = UUID.init().uuidString  // unique request id for this message, used in the ack
    
    private enum CodingKeys: CodingKey {
        case version, msgType, data, requestId
    }
    
    
    // Default constructor
    // Not used cause calling super.init can override msgType value
    init() {
        // empty on purpose
    }
    
    required init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        version = try container.decode(String.self, forKey: .version)
        msgType = try container.decode(MessageType.self, forKey: .msgType)
        data = try container.decode(String.self, forKey: .data)
        requestId = try container.decode(String.self, forKey: .requestId)
    }
    
    public func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: CodingKeys.self)
        try container.encode(version, forKey: .version)
        try container.encode(msgType, forKey: .msgType)
        try container.encode(data, forKey: .data)
        try container.encode(requestId, forKey: .requestId)
    }
}

/*
 * BroadcastMessage is used to broadcast a message to all connected clients
 */
class BroadcastMessage : Message {
    // response message
    var msgContent : String = ""   // Message to broadcast
    
    // coding keys enumeration used for JSON encoding/decoding
    private  enum CodingKeys: CodingKey {
        case msgContent
    }
    
    // set class variables
    private func initClassVars() {
        self.msgType = .broadcast
        msgContent = ""
    }
    
    // default constructor. Call the parent and set variables
    override init() {
        super.init()
        initClassVars()
    }
    
    // Constructor used to instantiate a class from JSON Data
    required init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        msgContent = try container.decode(String.self, forKey: .msgContent)
        try super.init(from: decoder)
    }
    
    // Method used to encode class to JSON
    override public func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: CodingKeys.self)
        try container.encode(msgContent, forKey: .msgContent)
        try super.encode(to: encoder)
    }
}

As you can see, when BroadcastMessage is converted to JSON, it is now correctly encoded.

I am now using the coding pattern below to achieve this functionality:

  • Create a private enum called CodingKeys that follows CodingKey. ]
  • Enter all class variables as part of the enumeration
  • Create custom encoders and decoders for the base class
  • In the subclass, define a new private enum called CodingKeys . I have marked both private so the compiler knows which variable to know in which function
  • Create the custom encoders
  • Encode the variables of the child class and then
  • Call the encoder / decoder of the parent class