Blog

This article is about installing Cisco Network Services Orchestrator (NSO) on Linux platform and connecting one southbound network device, a Cisco CSRv router, to it. All the demonstration will take place on the EVE-NG platform, using as an example the topology in the figure below. At the end of the article you will find a link to download the lab for EVE-NG.

NSO supports OS X and Linux platforms. For this example you can use a Linux VM or a Docker node, all depends on taste and the EVE version you have (EVE Community don’t support Docker). I will use Docker eve-gui-server node because it’s really fast and i don’t have to install any other software requirements of NSO (Java JDK-7.0 or higher and either Python 2.7.5 or Python3.4).

Note: you can download a ready to use Ubuntu image for EVE-NG from the following this link.

Installing NSO

If you are entitled, any version of NSO can be downloaded from Cisco Software Download page. The good news for all who don’t have a support contract is that Cisco DEVNET is offering NSO 5.1.0 free to download, just follow this link.

After NSO install package is downloaded in /opt directory (you can choose /home for example) it’s ready to be unpacked. NSO is delivered with a cryptographic signature to make it possible to verify that the installation has not been altered or corrupted. For this check to work, your Linux/Docker machine needs a working internet connection to be able to download CA certificate from cisco.com (http://www.cisco.com/security/pki/certs/crcam2.cer). Since i’m not in a production environment i will go ahead and execute the installation package, using:

root@DockerNSO:/#cd /opt
root@DockerNSO:/opt# sh nso-4.6.4.linux.x86_64.signed.bin --skip-verification
Unpacking...

If we have a look to /opt directory content, we can see five new files, which are:
– cisco_x509_verify_release.py – this is a python program that can be used to verify the 3-tier x.509 certificate chain and signature.
– nso-4.6.4.linux.x86_64.installer.bin – NSO installer
– nso-4.6.4.linux.x86_64.installer.bin.signature – Signature generated for the NSO image
– tailf.cer – Cisco signed public certificate used to verify the signature

NSO can be installed in two modes:
Local install – used for development, evaluation, PoC and lab purposes;
System install – this option is for production and system-wide deployment in a central location.

The command format to start local installation is:
$ sh nso-VERSION.OS.ARCH.installer.bin <Destination Directory>

I go to install nso in /opt/nso-4.6.4 with the following command:

root@DockerNSO:/opt# sh nso-4.6.4.linux.x86_64.installer.bin nso-4.6.4
INFO  Using temporary directory /tmp/ncs_installer.4008 to stage NCS installation bundle
INFO  Unpacked nso-4.6.4 in /opt/nso-4.6.4
INFO  Found and unpacked corresponding DOCUMENTATION_PACKAGE
INFO  Found and unpacked corresponding EXAMPLE_PACKAGE
INFO  Generating default SSH hostkey (this may take some time)
INFO  SSH hostkey generated
INFO  Environment set-up generated in /opt/nso-4.6.4/ncsrc
INFO  NCS installation script finished
INFO  Found and unpacked corresponding NETSIM_PACKAGE
INFO  NCS installation complete

In nso-4.6.4 directory we have many usefull things: doc directory which is full of useful documentation and different NSO documents; src is where you can find NSO source Yang and Python code; examples.ncs directory contains many example projects that you can use for practicing or as a base for your projects.

If we look at the installation output, we notice that the installer creates in /opt/nso-4.6.4 a shell script file named ncsrc. This script should be used to set the environment variables and commands. I will source this file to have all setings for my shell.

root@DockerNSO:/opt# source nso-4.6.4/ncsrc

It’s a good ideea to add this script in user owned .bashrc file because it will be sourced any time the Bash shell is started interactively (in my case, .bashrc file is located in /root directory and using a simple command i can forget about sourcing ncsrc).

root@DockerNSO:/opt# echo "source /opt/nso-4.6.4/ncsrc" >> /root/.bashrc

Now i have to use ncs-setup utility to create a runtime directory for NSO where is keeping its stuff (database, state files, logs and others), and i will name it ncs-run:

root@DockerNSO:/opt# ncs-setup --dest ncs-run

To start NSO i have to make sure i am in the deployment directory, because NSO reads config files from that place:

root@DockerNSO:/opt# cd ncs-run
root@DockerNSO:/opt/nso-run# ncs

No message is displayed when ncs startup succeeds, to get more details ( information about server version, loaded modules, etc.) we can use the --status parameter.

Now, we have two modes of interaction with NSO, CLI or WebUI. WebUI cand be accessed on 127.0.0.1:8080 with user admin password admin.

For console access is the same user/password combination, and we can use either of the following:

# ncs_cli -u admin
or
# ssh -l admin -p 2024 127.0.0.1

If it’s not specified otherwise, ncs_cli and ssh uses the username that are running the shell as, in my case root which is not a valid user for NSO.

NSO shell has two styles of interaction: Juniper and Cisco IOS XR. We can switch between them inside of the shell, using switch cli command or we can start ncs_cli with a parameter (-J for Juniper or -C for Cisco XR). By default, ncs_cli is starting in Juniper mode, we can switch between modes with switch cli command.

root@DockerNSO:/opt/nso-run# ncs_cli -u admin
admin connected from 127.0.0.1 using console on DockerNSO
admin@ncs> switch cli
admin@ncs#

To exit CLI type exit. To stop NSO daemon use ncs with --stop option.

root@DockerNSO:/opt/nso-run#ncs --stop

The main log file is located in /opt/nso-run/logs/ncs.log. To delete all log files and restore to initial configuration, use ncs-setup with the --reset option.

root@DockerNSO:/opt/nso-run# ncs-setup --reset

Is important to understand that NSO is a representation of the network devices and network services and NSO CLI it’s the northbound interface to that representation. Do not confuse NSO CLI with a shortcut to the end device CLI. NSO supports a mix of vendors and device interfaces and at the same time provides the northbound CLI to them.

Installing Network Element Drivers (NEDs)

What are NEDs? I will not explain in detail here. Basically, NEDs helps NSO to communicate with southbound devices (network elements) and is model driven. If you want to find more about NEDs, i recommend to read the documentation installed by NSO (doc directory) or this page from Cisco DEVNET.

NEDs distributed with the installed NSO are located in directory $NCS_DIR/packages/neds/. In $NCS_DIR/packages/services are Open Stack services files and $NCS_DIR/packages/tools contains Discovery Module service.

root@DockerNSO:/opt# ls $NCS_DIR/packages/neds/ 
a10-acos  cisco-ios  cisco-iosxr  cisco-nx  dell-ftos  juniper-junos

For this lab we need only cisco-ios network element driver. This directory must be copied (a softlink can be used) from NSO installation directory to the project directory.

root@DockerNSO:/opt# cp -r $NCS_DIR/packages/neds/cisco-ios ncs-run/packages/
root@DockerNSO:/opt# ls ncs-run/packages/
cisco-ios

Let’s go to NSO CLI and check and reload packages.

root@DockerNSO:/opt# cd ncs-run/
root@DockerNSO:/opt/ncs-run# ncs_cli -C -u admin
admin connected from 127.0.0.1 using console on DockerNSO
admin@ncs# packages reload 
reload-result {
    package cisco-ios
    result true
}
admin@ncs# show packages package package-version
             PACKAGE  
NAME         VERSION  
----------------------
cisco-ios    3.0.0.3

Adding CSR router to NSO

Devices can be added to NSO manually, using cloning, templates, bulk upload, device discovery, PnP, etc. For now i will choose manual mode.

To add a device manually i have to specify, at least, few things which includes: IP address, device type, NED to be used, protocol to access device and authentication group.

Let’s go in config mode and add admin user with password cisco to CSRAuthGroup. This user is already defined on the router.

admin@ncs(config)# devices authgroups group CSRAuthGroup
admin@ncs(config-group-CSRAuthGroup)# umap admin remote-name admin remote-password cisco remote-secondary-password cisco
admin@ncs(config-group-CSRAuthGroup)# top

To see uncommited configuration i use show configuration command. remote-secondary-password defines enable secret password.

admin@ncs(config)# show configuration
devices authgroups group CSRAuthGroup
 umap admin
  remote-name               admin
  remote-password           $8$iA8nNQyS1Jl40e84q3jAGePzFt0y/tMw5P1GDtVsmfA=
  remote-secondary-password $8$aHt42VS9RdHmVJJXRETS1EiY99OQvvUqnD9Wsz2CVk8=

To commit the config, i use commit command.

admin@ncs(config)# commit 
Commit complete.

Now i have authentication parameters defined, i can add the device.

admin@ncs(config)# devices device CSR-1 address 10.0.1.2
admin@ncs(config-device-CSR-1)# device-type cli ned-id cisco-ios protocol ssh
admin@ncs(config-device-CSR-1)# authgroup CSRAuthGroup
admin@ncs(config-device-CSR-1)# state admin-state unlocked
admin@ncs(config-device-CSR-1)# top
admin@ncs(config)#

You noticed that i used state admin-state unlocked which permits all changes to the device to be sent southbound. top has the same functionality like in Juniper CLI, allow to jump to the root of the configuration tree. Let’s see the device config and then commit the changes.

admin@ncs(config)# show config
devices device CSR-1
 address   10.0.1.2
 authgroup CSRAuthGroup
 device-type cli ned-id cisco-ios
 device-type cli protocol ssh
 state admin-state unlocked
 config
  no ios:service pad
  no ios:ip domain-lookup
  no ios:ip http secure-server
  ios:ip source-route
!
admin@ncs(config)# commit
Commit complete.
admin@ncs(config)# exit

What is the status of synchronization between NSO and CSR router?

admin@ncs# devices check-sync device CSR-1
sync-result {
    device CSR-1
    result error
    info Failed to authenticate towards device CSR-1: Unknown SSH host key
}
admin@ncs# *** ALARM connection-failure: Failed to authenticate towards device CSR-1: Unknown SSH host key

That’s not so good, NSO don’t recognize the device ssh host key, because it’s not in the config section of the device.

admin@ncs# show running-config devices device CSR-1
devices device CSR-1
 address   10.0.1.2
 authgroup CSRAuthGroup
 device-type cli ned-id cisco-ios
 device-type cli protocol ssh
 state admin-state unlocked
 config
  no ios:service pad
  no ios:ip domain-lookup
  no ios:ip http secure-server
  ios:ip source-route

Let’s get the ssh host key directly from the device.

admin@ncs# devices device CSR-1 ssh fetch-host-keys 
result updated
fingerprint {
    algorithm ssh-rsa
    value dd:a8:44:70:ee:78:e5:28:25:c2:70:1f:c8:2c:f4:5c
}

Any change in the running config of the device?

admin@ncs# show running-config devices device CSR-1
devices device CSR-1
 address   10.0.1.2
 ssh host-key ssh-rsa
  key-data "AAAAB3NzaC1yc2EAAAADAQABAAABAQCMVFnP2e8ppWfF0cGOltA5Y7MG3g6mjDh+UniRAtrZ\nw3+/uuscnly0sHW5mEcYvoEGKuZZAjmMaf1qdLDW4Ik/BhUlhU/QpJjKOTXozKqQI+F082Cp\nUysaGuWX9rQwvvsF8m6t/wBv4EAqjAw2b/WQW3O3jLhxN8UiV43vyeR2vXaJE7lqmW+k3LlG\njPJ+qJOPHQIdX0/F1m+ecO7fImu9D1bVdomGWLHcOq4/raoDj830DQsva5dfOwAQJuIwIJNG\nt5ITy2GYEhYycrfUlhLwhnU/yjxgKQ1qovX4EHfoN28nMKwqV3bZY8Vs+jtBKF2LkL8RE2HU\ntO0ANX3Om477"
 !
 authgroup CSRAuthGroup
 device-type cli ned-id cisco-ios
 device-type cli protocol ssh
 state admin-state unlocked
 config
  no ios:service pad
  no ios:ip domain-lookup
  no ios:ip http secure-server
  ios:ip source-route

I can see that ssh key was added to device config, so i can check synchronization again

admin@ncs# devices check-sync device CSR-1
sync-result {
    device CSR-1
    result unknown
}

Result unknown means that NSO doesn’t have a record of a transaction id with CSR-1, and it doesn’t know if it’s out of sync or not. We can add the running config of the device to NSO using sync-from.

admin@ncs# devices sync-from device CSR-1
sync-result {
    device CSR-1
    result true
}

Now the result is true, so transaction succeeded. We can use again check-sync.

admin@ncs# devices check-sync            
sync-result {
    device CSR-1
    result in-sync
}

And now i can see CSR-1 config in NSO.

admin@ncs# show running-config devices device CSR-1
devices device CSR-1
 address   10.0.1.2
 ssh host-key ssh-rsa
  key-data "AAAAB3NzaC1yc2EAAAADAQABAAABAQCMVFnP2e8ppWfF0cGOltA5Y7MG3g6mjDh+UniRAtrZ\nw3+/uuscnly0sHW5mEcYvoEGKuZZAjmMaf1qdLDW4Ik/BhUlhU/QpJjKOTXozKqQI+F082Cp\nUysaGuWX9rQwvvsF8m6t/wBv4EAqjAw2b/WQW3O3jLhxN8UiV43vyeR2vXaJE7lqmW+k3LlG\njPJ+qJOPHQIdX0/F1m+ecO7fImu9D1bVdomGWLHcOq4/raoDj830DQsva5dfOwAQJuIwIJNG\nt5ITy2GYEhYycrfUlhLwhnU/yjxgKQ1qovX4EHfoN28nMKwqV3bZY8Vs+jtBKF2LkL8RE2HU\ntO0ANX3Om477"
 !
 authgroup CSRAuthGroup
 device-type cli ned-id cisco-ios
 device-type cli protocol ssh
 state admin-state unlocked
 config
  ios:version  16.9
  no ios:service pad
  ios:service timestamps debug datetime msec
  ios:service timestamps log datetime msec
  ios:platform console serial
  no ios:platform punt-keepalive disable-kernel-core
  ios:hostname CSR-1
  ios:enable secret 5 $1$L5bT$D7GdWsXeDtf60HYfKXibe.
  ios:username admin privilege 15 secret 5 $1$k0ev$0nvL1TlkcjzruWRWihPrL.
  no ios:ip domain-lookup
  ios:ip forward-protocol nd
  ios:ip http authentication local
  ios:ip http server
  ios:ip http secure-server
  ios:ip source-route
  ios:interface GigabitEthernet1
   negotiation auto
   description Link to NSO
   ip address 10.0.1.2 255.255.255.0
  exit
  ios:interface GigabitEthernet2
   negotiation auto
   no ip address
   shutdown
  exit
  ios:interface GigabitEthernet3
   negotiation auto
   no ip address
   shutdown
  exit
  ios:interface GigabitEthernet4
   negotiation auto
   no ip address
   shutdown
  exit
  ios:interface Loopback1
   ip address 1.1.1.1 255.255.255.255
  exit
  ios:control-plane
  !
  ios:line vty 0 4
   login local
   transport input ssh
  !
  ios:multilink bundle-name authenticated
  ios:spanning-tree extend system-id
  ios:crypto pki trustpoint TP-self-signed-1690426881
   enrollment selfsigned
   revocation-check none
   rsakeypair       TP-self-signed-1690426881
   subject-name     cn=IOS-Self-Signed-Certificate-1690426881
  !
  ios:crypto pki certificate chain TP-self-signed-1690426881
   certificate self-signed 01 "\n this part was removed for brevity \tquit"
  !
  ios:license udi pid CSR1000V sn 9I9K4TFF6E5
 !
!
admin@ncs# 

Let’s add an new Loopback interface to CSR-1 using NSO CLI.

admin@ncs# config 
Entering configuration mode terminal
admin@ncs(config)# devices device CSR-1
admin@ncs(config-device-CSR-1)# config 
admin@ncs(config-config)# ios:interface Loopback 2
admin@ncs(config-if)# description CONFIG_FROM_NSO
admin@ncs(config-if)# ip address 2.2.2.2 255.255.255.255
admin@ncs(config-if)# top
admin@ncs(config)#
admin@ncs(config)# show config
devices device CSR-1
 config
  ios:interface Loopback2
   description CONFIG_FROM_NSO
   ip address 2.2.2.2 255.255.255.255
  exit
 !
!
admin@ncs(config)# commit and-quit
Commit complete.

As commit has succeeded, let’s go to the router CLI and see what we have there.

*%LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback2, changed state to up
*%SYS-5-CONFIG_I: Configured from console by admin on vty0 (10.0.1.1)

I have two notification logs, one is about Loopback 2 interface and the other one about the source of configuration, admin from 10.0.1.1, which is NSO. Now, let’s use a few show commands to confirm status of Loopback 2.

CSR-1#show ip interface brief 
Interface              IP-Address      OK? Method Status                Protocol
GigabitEthernet1       10.0.1.2        YES manual up                    up      
GigabitEthernet2       unassigned      YES unset  administratively down down    
GigabitEthernet3       unassigned      YES unset  administratively down down    
GigabitEthernet4       unassigned      YES unset  administratively down down    
Loopback1              1.1.1.1         YES manual up                    up      
Loopback2              2.2.2.2         YES manual up                    up 

CSR-1#show interfaces description 
Interface                      Status         Protocol Description
Gi1                            up             up       Link to NSO
Gi2                            admin down     down     
Gi3                            admin down     down     
Gi4                            admin down     down     
Lo1                            up             up       
Lo2                            up             up       CONFIG_FROM_NSO     

All looks right, NSO has deployed our configuration to the CSR-1 router.

Lab Resources

Links
NSO Documentation
Getting NSO
NSO Basics by Cisco IT (4 Labs)
NSO usage for different users (4 Labs)
The NSO Developer Hub
NSO on GitHub
Sandboxes on NSO