“Hey Juniper, How’s the security state of my network?”

“Hey Juniper, How’s the security state of my network?”

Most of us probably have asked Siri, Alexa, Google some questions and answers over the past years; Smart assistants are one of those new interactions. And honestly, not that many questions you asked were probably related to networks, were they? And yet, after hearing Juniper present at Security Field Day 3, these type of questions might in the near future become a normal way to allow you to interact with the network. Sounds like something from Star Trek, and is just science fiction? I beg to differ for a number of reasons. 

Before I dive into Juniper’s security strategy, let’s first review the current state of Artificial Intelligence (AI) and Machine Learning (ML) as a concept. To do that, I will explain what happens when you ask Siri to do something or a question like what is the date?.  

First of all, after you’ve asked the question, that audio bit (which is just a record of frequencies) is guessed into a sentence. It is a form of translation, where Machine Learning is used to do well-calculated guesses in what kind of word you tried to pronounce. And I can tell you that if English is not your native tongue, some weird word-guesses come out.

 

Once there is a digital sentence, another process is used to narrow down your digital sentence into a single pattern-matched result. This allows you to say things like “create a meeting on date X, Y or z”.
The model uses the pattern “create a meeting” for the match and the other values are stored in variables.

After a pattern has been matched, the code corresponding to that pattern is executed. The result of that code will be a string/sentence. That sentence is then being translated to audio using Text-To-Speech and read out loud to you.

This is a perfect flow for a single sentence / command /question. But many of these smart assistents start to show real issues once you start to create related questions, like you can do with a toddler.  And that is because the flow I described above does not have that much space for relaying context between flows. So most models in these smart assistents are built / designed for a single task, and if you put them in sequence, it appears as if the assistent is “smart”. But it is not (yet).

Yes, developments go fast, and I heard last year in San Jose a great story of self-learning models and also the core faults that are inside these models because of the way the models are designed and built. But that is a completely other discussion. 

But if AI/ML is not that far ahead yet, how would you then be able to ask Juniper about the security state of the network? That how is one of the things Juniper told at Security Field Day 3.

They leverage the power of AI/ML (and that is to have more answers than just a yes or now) in their security solutions for only a predefined, supervised, set of options. Juniper uses models (they call them supervised models) to analyse metadata about the flows in your network to detect malware and other anomalies in your network, just like other network and security vendors have done. Juniper has adopted AI/ML on detecting anomalies or other weird behaviour on your network. And that can help you a lot in gaining more visibility and control in your network.

Because with everything connected to the network, increased security by leveraging encryption, more data in the cloud, we as a human are reaching our limits in finding anomalies or odd behaviors by hand. A simple computer, using a properly tuned model, can find those anomalies much faster. And because that model does that, you can focus on determining whether that anomaly is really an anomaly of yet another new application on the network.

And if you would trust those reports blindly, you can of course automate actions, but personally I am always careful with those. Suppose it is the CEO’s smartphone, you might want to call the CEO first before you throw him off the network.

Personally I do believe firmly in that AI/ML can help you in your daily job in finding those odd behaviors that without them you’d never be able to spot and help you in providing a better, smoother, more stable and secure network. But you do need to keep in mind that it is still a computer that presents you information from a lot of data and that presentation will be biased.

But definitely a good thing that Juniper has started to embrace that principle across their portfolio. 
That can bring them quite a bit.

And to be able to ask Juniper about the security state of the network? That would be not that hard to implement, because JunOS has API’s from the start. Just build an application and integrate it with existing API’s for smart assistents. If you want to see a demo, just watch this video taken from a presentation where I asked Siri how many clients were connected to the network.

Cisco C9800-CL sits idle at GRUB Loading Stage2…

Cisco C9800-CL sits idle at GRUB Loading Stage2…

I have been using the Cisco Catalyst 9800-CL (Wireless Controller for cloud) for a while now. Recently, I accidentally powered off the wrong VMWare server, resulting in a wireless disruption. Priority 1 at home! And of course, just before I had a session with Shawn preparing for CiscoLive Barcelona…

After restarting the vSphere server, my C9800-CL wasn’t booting up, with a message: “GRUB Loading stage2… ” And it just sat there, for minutes..  Eventually, during the WebEx Call, I managed to fix it and got my controller back up and running. 

Steps to fix the issue

These are the steps that I used for fixing this issue.

First, power off the VM in vSphere. We need to change some settings in the BIOS.

Next, go and select “Edit Settings” of your VM and click “VM Options” at the top to view some advanced settings and click “Boot options” open. Change the Boot delay to “8000” milliseconds, so that you have enough time when you boot the VM.

Hit Save after you have changed the settings.

Just to make sure, open the settings of the VM again, and click open the first CD/DVD Drive. Check that there is an image named “_deviceImage-0.iso” and that it is connected at powerup.

When I used the vCenter convertor to move the VM off to a new server, I found that this iso wasn’t copied with the controller and it is needed.

Hit Save when you know the ISO image is there.

Follow the next steps to get the C9800-CL booting up again

  1. Open up the console of the VM in the browser (it saves you time)
  2. Power on the VM
  3. Once the Bios is shown, click in the console and hit “ESC
  4. The boot order menu is shown, like the image on the left
  5. Scroll down to highlight “CD-ROM Drive” 
  6. And hit “Enter
  7. Now the VM will boot normally and your controller will start as expected.

Summary

It seems that Grub (the bootloader on the first disk) is not configured correctly the C9800-CL, which leads to a VM / Appliance that is not booted because it cannot find any kernel to load. By selecting the CD image, the right bootloader is selected and the controller is started with the correct configuration. 

I do assume this is a caveat/bug in the Cloud version and will be fixed in a newer release. I do hope you can use this info to fix your C9800-CL deployment sooner. 

Swift, JSON Encoding/Decoding and subclasses

Swift, JSON Encoding/Decoding and subclasses

Over the past weeks I have been preparing for two CiscoLive Barcelona breakout sessions. In one of them I will give a brief demo and the other session where I will be covering parts of the Cisco Press book that I wrote. The preparation itself is not only about the slides, but also developing code that is to be used in the demo’s. These demo’s are built on iOS devices and run on some containers, so I have been writing that software in Swift, which is a beautiful and powerful programming language. One of my previous posts covers some principles of Swift. One really powerful feature is the easy capability to encode or decode data to the JSON format.  

If you want to have a class to be able to convert to and from a JSON format, just use the Codable protocol and you’re ready, see the code example below:

This code example defines a class message with variables for messageType (of type MessageType), requestId, which is a unique UUID string value, and a data variable which can contain any String. So let’s say I create a new message , called hello with the data “Hello there!” with the following code sample:

To convert this to JSON, this would only require a few lines of code:

The variable jsonData (of type Data) now contains a JSON-version of the earlier created message. Just to check the output, I can use the following commands to convert that data to String and output it in XCode’s Playground. 

Suppose you would like to extend our message class with a special broadcast message, where the message can be sent to a all endpoints.. You could add an optional broadcastContent variable to the message class and create a state machine to determine when to use that value. Another alternative is to leverage the power of object-oriented programming and create a new subtype, like the following code example:

So when you’d create a multicast message, like below, you’d expect that it would contain all attributes in the json file, right? Let’s check it out in Playground:

As you can see, the output does not contain all attributes of the broadcast message! It only contains the base message type class values. The msgContent variable is not included. It took me some time debugging and researching to figure out what happens. Swift bug SR-5431 and SR-4722  provide more details. Without going into those bugs, it comes down to the fact that as soon as you subclass a class that conforms to Codable, you need to override the default encode/decode methods and write your own. After some fiddling around, I have used the following code pattern to achieve that result.

As you can see, when BroadcastMessage is converted to JSON, it is now correctly encoded.

I am now using the coding pattern below to achieve this functionality:

  • Create a private enum called CodingKeys that follows CodingKey. ]
  • Enter all class variables as part of the enumeration
  • Create custom encoders and decoders for the base class
  • In the subclass, define a new private enum called CodingKeys . I have marked both private so the compiler knows which variable to know in which function
  • Create the custom encoders
  • Encode the variables of the child class and then
  • Call the encoder / decoder of the parent class 

Cisco C9800-CL sits idle at GRUB Loading Stage2…

Upgrading Firepower1010 to 6.5

The Cisco FirePower 1010 appliance (FP1010, successor to the ASA5506 which can run FTD 6.3 and higher) has finally become available. As I am relocating to a new home, it was time to replace my trusty 5506-X with the FP1010 and get a new fresh start with FTD. Since FTD 6.5 is just out, and it enables the switchports on the FP1010, it was time to upgrade the appliance. In this post I will share my method of upgrading the FP1010 to the latest version, 6.5. 

Time to get started with the upgrade. In this blog post I assume the FP1010 appliance has never been booted and has just been unboxed. You need to have the following items

  • Laptop with FTP/SCP/SFTP server (TFTP is possible, I had issues with USB); I used my MacBookPro for this
  • Laptop connected to the management interface of the FP1010
  • The upgrade image, in my case: cisco-ftd-fp1k.6.5.0-115.SPA

Once you have everything ready, the following steps can be used to upgrade the FP1010 appliance:

Firepower architecture

Firepower appliances are really a different platform to the trusty old ASA platform. One of the architectural differences is that the appliance is running FXOS as the operating system and the security services you want to run (FTD or ASA) are installed as an instance. I think the best to compare it with is VMWare and running virtual services. FXOS looks a lot in its command set to the NFVIS operating system that runs on the ENCS series. It is based on the UCS platform and uses quite a different CLI then you are familiar with in the ASA world. 

The larger appliances (FP4100 and FP9300) FXOS and the security instances are separated, which means that you first configure FXOS and then you can load the security instance on it. The smaller Firepower appliances, such as the FP2100, FP1100 and the FP1000 series have FXOS and the security instance bundled in a single release. This means that you always run a specific FXOS system with a specific ASA or FTD version.

1.  Connect the console of the FP1010 to the laptop and power on the appliance
2.  Connect a network cable from the mgmt interface to your laptop

3.  Wait until the FP1010 is booted. Once it’s booted, the console will show:

firepower#

4.  Type the command “connect ftd” and run through the initial setup wizard. If you do not accept the EULA and run through the setup, somehow the network is not working as expected and you cannot download the software. And yes, that took me some hours to figure out…

You must accept the EULA to continue.Press <ENTER> to display the EULA:
 
End User License Agreement

Effective: May 22, 2017

*** SNIP***
Please enter 'YES' or press  to AGREE to the EULA: YES

System initialization in progress.  Please stand by.
You must change the password for 'admin' to continue.
Enter new password:
Confirm new password:
You must configure the network to continue.
You must configure at least one of IPv4 or IPv6.
Do you want to configure IPv4? (y/n) [y]: y
Do you want to configure IPv6? (y/n) [n]: n
Configure IPv4 via DHCP or manually? (dhcp/manual) [manual]:
Enter an IPv4 address for the management interface [192.168.45.45]:
Enter an IPv4 netmask for the management interface [255.255.255.0]:
Enter the IPv4 default gateway for the management interface [data-interfaces]:
Enter a fully qualified hostname for this system [firepower]:
Enter a comma-separated list of DNS servers or 'none' [208.67.222.222,208.67.220.220]:
Enter a comma-separated list of search domains or 'none' []:
If your networking information has changed, you will need to reconnect.

Setting DNS servers: 208.67.222.222 208.67.220.220
No domain name specified to configure.
Setting hostname as firepower
DHCP server is enabled with pool: 192.168.45.46-192.168.45.254. You may disable with configure network ipv4 dhcp-server-disable
Setting static IPv4: 192.168.45.45 netmask: 255.255.255.0 gateway: data on management0
Updating routing tables, please wait...
All configurations applied to the system. Took 3 Seconds.
Saving a copy of running network configuration to local disk.
For HTTP Proxy configuration, run 'configure network http-proxy'

Manage the device locally? (yes/no) [yes]: yes
Configuring firewall mode to routed


Update policy deployment information
    - add device configuration
Successfully performed firstboot initial configuration steps for Firepower Device Manager for Firepower Threat Defense.

5.  After the setup, the console will have a very empty prompt: “>” Now type exit The prompt will now look like firepower# 

6. This means you are now in FXOS , this looks like UCS CIMC software, so it is a bit different.
Enter the command scope firmware , the prompt will show

firepower /firmware
7. Check the IP address of your laptop and initiate the software download via the command structure

download image sftp://userid@iplaptop/path/to-image/cisco-ftd-fp1k.6.5.0-115.SPA

I have used

download image sftp://myuserid@192.168.45.46/Users/myuserid/Downloads/cisco-ftd-fp1k.6.5.0-115.SPA

The console will now prompt for your password and then it will initiate a download task:

firepower /firmware # download image scp://myuserid@192.1687.45.46:/Users/myuserid/Downloads/cisco-ftd-fp1k.6.5.0-115.SPA
Password:
Please use the command 'show download-task' or 'show download-task detail' to check download progress.

You can use the “show download-task detail” to show the details, which has output like

Download task:
File Name: cisco-ftd-fp1k.6.5.0-115.SPA
Protocol: Sftp
Server: 192.168.45.46
Port: 0
Userid: myuserId
Path: /Users/myuserId/Downloads
Downloaded Image Size (KB): 59264
Time stamp: 2019-10-07T06:48:09.268
State: Downloading
Status: Downloading the image
Transfer Rate (KB/s): 29632.000000
Current Task: downloading image cisco-ftd-fp1k.6.5.0-115.SPA from 192.168.45
.46(FSM-STAGE:sam:dme:FirmwareDownloaderDownload:Local)

However, if there is a failure, it will only show “failed“. I found out that the command

show event provides much more information, but requires a bit decoding. The following output is from a successful download:
Creation Time            ID       Code     Description
------------------------ -------- -------- -----------
2019-10-07T06:48:09.269     27339 E4195702 [FSM:STAGE:END]: (FSM-STAGE:sam:dme:F
irmwareDownloaderDownload:begin)
2019-10-07T06:48:09.269     27340 E4195703 [FSM:STAGE:END]: checking pending man
agement network config(FSM-STAGE:sam:dme:FirmwareDownloaderDownload:CheckPending
NetworkConfig)
2019-10-07T06:48:09.269     27341 E4195704 [FSM:STAGE:ASYNC]: downloading image
cisco-ftd-fp1k.6.5.0-115.SPA from 192.168.45.46(FSM-STAGE:sam:dme:FirmwareDownlo
aderDownload:Local)
But if there is a failure, it would look a bit more like this

 

2019-10-07T06:47:40.120     27329 E4195706 [FSM:STAGE:REMOTE-ERROR]: Result: end
-point-failed Code: ERR-DNLD-no-file Message: No such file#(sam:dme:FirmwareDown
loaderDownload:DeleteLocal)

It tells you it couldn’t find the file. The show event is quite handy.
Once the download is completed, the show detail command would look like this:

Download task:
    File Name: cisco-ftd-fp1k.6.5.0-115.SPA
    Protocol: Sftp
    Server: 192.168.45.46
    Port: 0
    Userid: nefkensp
    Path: /Users/nefkensp/Downloads
    Downloaded Image Size (KB): 1031174
    Time stamp: 2019-10-07T06:48:09.268
    State: Downloading
    Status: validating and unpacking the image
    Transfer Rate (KB/s): 32224.187500
    Current Task: unpacking image cisco-ftd-fp1k.6.5.0-115.SPA on primary(FSM-ST

8.  Now that the software is downloaded, it is time to validate if the package is available. Use the command show package to check for that:

firepower /firmware # show package
Name Package-Vers
--------------------------------------------- ------------
cisco-ftd-fp1k.6.4.0-102.SPA 6.4.0-102
cisco-ftd-fp1k.6.5.0-115.SPA 6.5.0-115

9.  Now as the package is available, let’s install it. Go to the subscope auto-install:

firepower /firmware # scope auto-install
firepower /firmware/auto-install # 
 

10.  and install the package via the install security-pack version command:

firepower /firmware/auto-install # install security-pack version 6.5.0-115 
The system is currently installed with security software package 6.4.0-102, which has:
   - The platform version: 2.6.1.133
   - The CSP (ftd) version: 6.4.0.102
If you proceed with the upgrade 6.5.0-115, it will do the following:
   - upgrade to the new platform version 2.7.1.107
During the upgrade, the system will be reboot

Do you want to proceed ? (yes/no):yes

This operation upgrades firmware and software on Security Platform Components
Here is the checklist of things that are recommended before starting Auto-Install
(1) Review current critical/major faults
(2) Initiate a configuration backup

Do you want to proceed? (yes/no):yes

Triggered the install of software package version 6.5.0-115
Install started. This will take several minutes.
For monitoring the upgrade progress, please enter 'show' or 'show detail' command. 

11. Now let’s wait for the upgrade or use the “show” command to check the status:

firepower /firmware/auto-install # show

Firmware Auto-Install:
    Package-Vers Oper State                   Upgrade State
    ------------ ---------------------------- -------------
    6.5.0-115    Scheduled                    Ready
firepower /firmware/auto-install #

12.  And after waiting for some 20-30 minutes, FTD has been upgraded. Congratulations!

Cisco C9800-CL sits idle at GRUB Loading Stage2…

Assigning a single IPv6 address to devices

I have been running IPv6 and IPv4 concurrently. At Cisco Live San Diego 2019  I shared some of my experiences with Jeffry Handal (I met him initially at CiscoLive Barcelona 2019)  and somewhow we ended up talking about IPv6 and how by default you receive multiple IPv6 addresses. To me, that was one of my frustrations, so my network is setup in such a way that it only assigns a single IPv6 address. It appears that such a setup is not very common. So I would like to share with you how my IPv6 network is configured.

My network consists of an ASA firewall (soon to be replaced with the FirePower 1010), a 3560 compact switch that acts as L3 switch, and a Catalyst 9800 Wireless Controller (yep, moved from Mobility Express to the Cat9k wireless IOS-XE). The figure below shows my network topology.

In this network setup, the 3560 acts as L3 switch and DHCP server (both IPv4 and IPv6). It is absolutely possible to use an external DHCP server and use helpers instead. But for my home network that is, well, not necessary. The configuration on the client VLAN is shown below:

interface Vlan300
 description clients
 ip address 192.168.1.1 255.255.255.0
 ipv6 address FE80::300 link-local
 ipv6 address 2001: db8:face:300::1/64
 ipv6 enable
 ipv6 nd prefix 2001:db8:face:300::/64 300 300 no-autoconfig
 ipv6 nd managed-config-flag
 ipv6 nd router-preference High
 ipv6 nd ra interval 30
 ipv6 dhcp server clients-300 rapid-commit
end

By setting the managed-config-flag and disabling auto-config on the prefix I effectively state that my switch is the only router and device allowed to assign and distribute IPv6 addresses. I effectively disable every auto-magic feature within IPv6 except DHCPv6. The configuration I use for that DHCPv6 server is defined below:

ipv6 dhcp database flash:dhcpv6-db
ipv6 dhcp pool clients-300
 address prefix 2001:db8:face:300::/64 lifetime 86400 86400
 link-address 2001:db8:face:300::/64
 dns-server 2620:119:35::35
 dns-server 2620:119:53::53
 domain-name clients.nefkens.net
!

Using this configuration all my devices (and yes, Jeffry told me that Android devices do not support DHCPv6 so go complain at Google for that) receive a single IPv6 address, as can be shown in the screen shot below.

Although it might not be common, it is very much possible to use DHCPv6 and only assign a single IPv6 address to each device. It will make your life for troubleshooting or looking at management systems, such as Firepower Management Center, DNA Center, or Syslog server a lot easier.

PoC on Network API’s

PoC on Network API’s

At Cisco Live 2018 Barcelona, Cisco Systems announced API’s to get network assurance data from Cisco DNA Center. The possibility to get information from the status of the network, its connected clients via an API is very powerful.

The power of network API’s started me thinking in what could be possible if you bring these network API’s to the software developer world, where using API’s (also known as frameworks) are as common as a simple if-statement.

Around the same time, Apple Systems announced a major upgrade of their Augmented Reality framework (Also a set of API’s which allows a developer to create a virtual reality overlay to a camera shot).

I talked with a System Engineer working with DNAC in Barcelona and shortly afterward of that idea and we agreed that I could demonstrate an application that showcases such possibilities in one of his sessions in 2019. 

And that plan came true. I was a speaker at a Cisco Live breakout session in Barcelona this year and demonstrated this application. I will write down that experience at a later time (if there is interest). But since that demonstration, I have received a number of requests to either publish the application or make that video available for demonstration purposes. So here it is.

PoC: Visual Wireless Troubleshooting App

The troubleshooting of a wireless network can be quite difficult, because of its dynamics and specifically, remote troubleshooting is challenging, checking out your laptop, determining to which Access Point it is connected and which clients are connected.

I created an application that uses a number of new emerging technologies, such as machine learning, augmented reality and of course the Network Intent API’s to demonstrate how an application can make that life easier.

The flow for the user is quite simple.

  1. Start the app
  2. Point the camera to a Cisco Access Point
  3. Machine Learning / Image Recognition will recognize that it is an Access Point
  4. Determine the access point name (on Apple iOS is that more than Android)
  5. Go to DNA Center and get all clients connected to that AP
  6. And show that in an AR experience

I built this app and demonstrated it at Cisco Live 2019 in Barcelona and used Cisco Live’s own DNA Center for the data. The screen recording I made for the app is shown below.

As you can see, I point my camera up to the ceiling, the AP is recognized and the client data is retrieved. If there is interest, I can share more insights in how I connect to DNA Center (using Swift) and how to get that data. As said this is just a proof of concept and a lot more can be built if you bring the programmable network to the software engineering world!

If you have some ideas, please share them. Who knows, somebody might build your dream App, or.. Start coding on your own. Check out Cisco’s DevNet for network programming API’s and Apple’s Swift Playground  and start coding!