OK, this is the continuation of the two articles here. The first is me Recent NETCONF tutorials , followed by my very old project (then in Java) Visualize the network topology using SNMP information named "hellorote" . So this is the resurrection of a very old idea, just using newer methods and tools. But the first is the preface about visualization.

Preface -- the author's experience in the visual use of network infrastructure

Well, as far as I'm concerned, automated network visualization or documents have never really been the main source of documents. Wherever I see, we still maintain manually created maps with version control, try to keep them up to date during the change process, and so on... This is because human authors can give the context of the map, such as by object Or part of a building or legal organization mapped office network. Look at the figure below, which is the difference between manual map and automatic map in most general network modeling tools.

Human and Computer generated network diagram

Now, in order not to completely kill the focus of completing this tutorial, I think the problem is that most visualization tools on the market are general-purpose products. I mean, they have general algorithms and only follow the "expectations" of suppliers in advance There are not many ways to add additional code / logic to suppliers without paying $$$to adapt them to your visual context. Therefore, from my point of view, for any network infrastructure maintenance organization (such as myself working or self consulting), the solution is to use available components (libraries, algorithms) Build visualization tools internally instead of proposing generic vendor software if they can't be mass customized (I haven't found such a vendor yet).

So what will we build in this tutorial?

Simply, we'll build something like this, which is the middle ground between the two extremes, or at least for my small spine leaf network( Live demo link here):

This is not the best visualization in the world (I don't claim it), the point here is to try to show you that building something yourself is not as difficult as it seems with a little script.

Part I LAB topology and prerequisites

We will continue where we left in the NETCONF tutorial last time and iterate over the same labs and requirements again.

A. Laboratory topology

As simple as last time, my development laptop can access two comware7 switches through SSH/IP. These switches have some LLDP neighbors on the uplink and server downlink.

LAB Topology is two active comware7 switches with IP management access

Note: Yes, the server uses LLDP. From my point of view, it is a good practice to run it. On Windows and Linux, it can be enabled by clicking the / command, and it can also be used by the management program in the DC, such as the distributed switch setting on ESX / vmWare. This is an enabled check box.

B. Installed python + Libraries

I will use the python 2.7 interpreter, so download it for your system (linux is preferred for easy installation) and download / install the ncclient library using pip and HP networking pyhpecw7 library using the following commands:

# Install ncclient
pip install ncclient

# Install HPN's pyhpecw7 library
git clone https://github.com/HPENetworking/pyhpecw7.git
cd pyhpecw7
sudo python setup.py install

C. (optional) Web development tool plug-in

When we start developing visualization, we will use simple HTML/CSS + Javascript. You only need a notepad to deal with this problem, but in order to troubleshoot, I really suggest you search the "Web development tool" plug-in on the browser or at least understand where your specific browser has a "console" View so that you can see what Javascript prints to the console as a debug message when a problem occurs.

Part 2. Example Python script for pulling LLDP neighbors to JSON mapping

Using the knowledge you gained in my previous NETCONF tutorial, you should be able to understand what happened here, so I'll put the whole code here and provide some explanations. But I won't introduce it line by line, because this is not a Python tutorial (other pages). The code should be:

Input: the "DEVICES.txt" file needs to exist nearby, which has a list of IP / host names to access, one host per line. For example, in my DEVICES.txt:

AR21-U12-ICB1
AR21-U12-ICB2

OUTPUT: the script will then generate three JSON files, which are:

• graph.json  – The main topology file, which will save all graphical nodes (devices) and links (interfaces) in JSON format. We will use it as visual input
• no_neighbor_interfaces.json  – This is an additional JSON file. I decided to create it to provide a quick list of interfaces in the "UP" operation state, but there are no LLDP neighbors behind them. This is an "unknown factor" risk, which I want to realize in my visualization exercise
• neighborships.json  – This describes the interface list for each device and the LLDP neighbors behind each interface.

Script source:

#!/bin/python
from pyhpecw7.comware import HPCOM7
from pyhpecw7.features.vlan import Vlan
from pyhpecw7.features.interface import Interface
from pyhpecw7.features.neighbor import Neighbors
from pyhpecw7.utils.xml.lib import *
import yaml
import pprint
import json
import re

##########################
# CONFIGURATION OF ACCESS
##########################
USER="admin"
PASS="admin"
PORT=830

#########################################################
# REGULAR EXPLRESSIONS FOR MATCHING PORT NAMES TO SPEEDS
# NOTE: This is used in visuzation later to color lines
#########################################################
LINK_SPEEDS = [ ("^TwentyGigE*","20"), 
                ("^FortyGig*","40") , 
                ("^Ten-GigabitEthernet*","10") , 
                ("^GigabitEthernet*","1")  ]

#########################################################
# REGULAR EXPLRESSIONS FOR MATCHING DEVICES HIERARHY
# E.g. Access layer switches have "AC" in their name
# or aggregation layer devices have "AG" in their names
#########################################################
NODE_HIERARCHY = [ ('.+ICB.*',"2"), 
                   ('^[a-zA-Z]{5}AG.*',"3"), 
                   ('^[a-zA-Z]{5}AC.*',"2"), 
                   ('^[a-zA-Z]{5}L2.*',"4") ]

####################
# Connection method
####################
def connect(host,username,password,port):
  args = dict(host=host, username=username, password=password, port=port)           
  print("Connecting " + host)
  # CREATE CONNECTION
  device = HPCOM7(**args)
  device.open()
  return device

################################
# Returns RAW python Dictionary 
# with Neighbor NETCONF details
#################################
def getNeighbors(device):
  print 'getNeighbors'
  neighbors = Neighbors(device)
  neigh_type=dict(default='lldp', choices=['cdp', 'lldp'])
  response = getattr(neighbors, "lldp")
  results = dict(neighbors=response)
  clean_results = list()
  
  for neighbor in results['neighbors']:
    if str(neighbor['neighbor']) == "None" or str(neighbor['neighbor']) == "":
      print("Removing probably bad neighbor \""+ str(neighbor['neighbor']) + "\"");
    else:
      clean_results.append(neighbor)
       
  return clean_results
  
###############################################
# Takes RAW Dictionary of Neighbors and returns
# simplified Dictionary of only Neighbor nodes 
# for visuzation as a node (point)
#
# NOTE: Additionally this using RegEx puts layer
# hierarchy into the result dictionary
###############################################  
def getNodesFromNeighborships(neighborships):
  print "getNodesFromNeighborships:"
  nodes = {'nodes':[]}
  for key,value in neighborships.iteritems():
    print("Key:" + str(key) + ":")

    '''
    PATTERNS COMPILATIOn
    '''
    print ("Hostname matched[key]: " + key)
    group = "1" # for key (source hostname)
    for node_pattern in NODE_HIERARCHY:
      print ("Pattern: " + node_pattern[0]);
      pattern = re.compile(node_pattern[0]);
      if pattern.match(key):
        print("match")
        group = node_pattern[1]
        break
    print("Final GROUP for key: " + key + " is " +group)

    candidate = {"id":key,"group":group}
    if candidate not in nodes['nodes']:
      print("adding")
      nodes['nodes'].append(candidate)
      
    for neighbor in value:
      print("neighbor: " + str(neighbor['neighbor']) + ":")
      '''
      PATTERNS COMPILATIOn
      '''
      print ("Hostname matched: " + neighbor['neighbor'])
      group = "1"
      for node_pattern in NODE_HIERARCHY:
        print ("Pattern: " + node_pattern[0]);
        pattern = re.compile(node_pattern[0]);
        if pattern.match(neighbor['neighbor']):
          print("match")
          group = node_pattern[1]
          break
      print("Final GROUP for neighbor: " + key + " is " +group)
            
      
      candidate2 = {"id":neighbor['neighbor'],"group":group}
      if candidate2 not in nodes['nodes']:
        print("adding")
        nodes['nodes'].append(candidate2)
    
  return nodes
    
###############################################
# Takes RAW Dictionary of Neighbors and returns
# simplified Dictionary of only links between 
# nodes for visuzation later (links)
#
# NOTE: Additionally this using RegEx puts speed
# into the result dictionary
###############################################    
def getLinksFromNeighborships(neighborships):
  print "getLinksFromNeighborships:"
  
  links = {'links':[]}
  for key,value in neighborships.iteritems():
    print(str(key))
    for neighbor in value:

      '''
      PATTERNS COMPILATIOn
      '''      
      print ("Interface matched: " + neighbor['local_intf'])
      speed = "1" # DEFAULT
      for speed_pattern in LINK_SPEEDS:
        print("Pattern: " + speed_pattern[0])
        pattern = re.compile(speed_pattern[0])
        
        if pattern.match(neighbor['local_intf']):
          speed = speed_pattern[1] 
      
      print("Final SPEED:" + speed)
      
      links['links'].append({"source":key,"target":neighbor['neighbor'],"value":speed})
  
  return links

##############################################
# Filters out links from simplified Dictionary 
# that are not physical 
# (e.g Loopback or VLAN interfaces)
#
# NOTE: Uses the same RegEx definitions as
# speed assignment
##############################################  
def filterNonPhysicalLinks(interfacesDict):

  onlyPhysicalInterfacesDict = dict()
  
  print "filterNonPhysicalLinks"
  for key,value in interfacesDict.iteritems():
    print("Key:" + str(key) + ":")
    onlyPhysicalInterfacesDict[key] = [];
    
    for interface in value:
     
      bIsPhysical = False;
      for name_pattern in LINK_SPEEDS:
        pattern = re.compile(name_pattern[0])
        
        if pattern.match(interface['local_intf']):
          bIsPhysical = True;
          onlyPhysicalInterfacesDict[key].append({"local_intf":interface['local_intf'],
                                                  "oper_status":interface['oper_status'],
                                                  "admin_status":interface['admin_status'],
                                                  "actual_bandwith":interface['actual_bandwith'],
                                                  "description":interface['description']})
          break;          
      
      print(str(bIsPhysical) + " - local_intf:" + interface['local_intf'] + " is physical.")
          

  return onlyPhysicalInterfacesDict;
  
##############################################
# Filters out links from simplified Dictionary 
# that are not in Operational mode "UP" 
############################################## 
def filterNonActiveLinks(interfacesDict):

  onlyUpInterfacesDict = dict()
  
  print "filterNonActiveLinks"
  for key,value in interfacesDict.iteritems():
    print("Key:" + str(key) + ":")
    onlyUpInterfacesDict[key] = [];
    
    for interface in value:
      if interface['oper_status'] == 'UP':     
        onlyUpInterfacesDict[key].append({"local_intf":interface['local_intf'],
                                          "oper_status":interface['oper_status'],
                                          "admin_status":interface['admin_status'],
                                          "actual_bandwith":interface['actual_bandwith'],
                                          "description":interface['description']})   
        print("local_intf:" + interface['local_intf'] + " is OPRATIONAL.")
          
  return onlyUpInterfacesDict;  
  
################################################
# Takes RAW neighbors dictionary and simplified 
# links dictionary and cross-references them to 
# find links that are there, but have no neighbor
################################################  
def filterLinksWithoutNeighbor(interfacesDict,neighborsDict):

  neighborlessIntlist = dict()
  
  print "filterLinksWithoutNeighbor"
  for devicename,neiInterfaceDict in neighborships.iteritems():
    print("Key(device name):" + str(devicename) + ":")
    
    neighborlessIntlist[devicename] = []
    
    for interface in interfacesDict[devicename]:
      bHasNoNeighbor = True
      for neighbor_interface in neiInterfaceDict:
        print("local_intf: " + interface['local_intf'] 
           + " neighbor_interface['local_intf']:" + neighbor_interface['local_intf'])
        if interface['local_intf'] == neighbor_interface['local_intf']:
          # Tries to remove this interface from list of interfaces
          #interfacesDict[devicename].remove(interface)
          bHasNoNeighbor = False
          print("BREAK")
          break;
          
      if bHasNoNeighbor:
        neighborlessIntlist[devicename].append(interface)
        print("Neighborless Interface on device: " + devicename + " int:" + interface['local_intf'])  
            
  return neighborlessIntlist;    
    
###########################
# Collects all Interfaces
# using NETCONF interface
# from a Device
# 
# NOTE: INcludes OperStatus
# and ActualBandwidth and  
# few other parameters 
###########################
  
def getInterfaces(device):
  print 'getInterfaces'

  E = data_element_maker()
  top = E.top(
      E.Ifmgr(
          E.Interfaces(
            E.Interface(
            )
          )
      )
  )
  nc_get_reply = device.get(('subtree', top))
  
  intName = findall_in_data('Name', nc_get_reply.data_ele)
  ## 2 == DOWN ; 1 == UP
  intOperStatus = findall_in_data('OperStatus', nc_get_reply.data_ele)
  ## 2 == DOWN ; 1 == UP
  intAdminStatus = findall_in_data('AdminStatus', nc_get_reply.data_ele)
  IntActualBandwidth = findall_in_data('ActualBandwidth', nc_get_reply.data_ele)
  IntDescription = findall_in_data('Description', nc_get_reply.data_ele)
  
  deviceActiveInterfacesDict = []
  for index in range(len(intName)):
  
    # Oper STATUS
    OperStatus = 'UNKNOWN'
    if intOperStatus[index].text == '2':
      OperStatus = 'DOWN'
    elif intOperStatus[index].text == '1':
      OperStatus = 'UP'
      
    # Admin STATUS
    AdminStatus = 'UNKNOWN'
    if intAdminStatus[index].text == '2':
      AdminStatus = 'DOWN'
    elif intAdminStatus[index].text == '1':
      AdminStatus = 'UP'    
         
    deviceActiveInterfacesDict.append({"local_intf":intName[index].text,
                                       "oper_status":OperStatus,
                                       "admin_status":AdminStatus,
                                       "actual_bandwith":IntActualBandwidth[index].text,
                                       "description":IntDescription[index].text})  
  
  return deviceActiveInterfacesDict  
  
   
###########################
# MAIN ENTRY POINT TO THE 
# SCRIPT IS HERE
###########################  
if __name__ == "__main__":
  print("Opening DEVICES.txt in local directory to read target device IP/hostnames")
  with open ("DEVICES.txt", "r") as myfile:
    data=myfile.readlines()
    
    '''
    TRY LOADING THE HOSTNAMES 
    '''
    print("DEBUG: DEVICES LOADED:")
    for line in data:
      line = line.replace('\n','')
      print(line)
      
    
    #This will be the primary result neighborships dictionary
    neighborships = dict()
    
    #This will be the primary result interfaces dictionary
    interfaces = dict()    
    

    '''
    LETS GO AND CONNECT TO EACH ONE DEVICE AND COLLECT DATA
    '''
    print("Starting LLDP info collection...")
    for line in data:
      #print(line + USER + PASS + str(PORT))
      devicehostname = line.replace('\n','')
      device = connect(devicehostname,USER,PASS,PORT)
      if device.connected: print("success")
      else: 
        print("failed to connect to " + line + " .. skipping");
        continue;

      ###
      # Here we are connected, let collect Interfaces
      ###
      interfaces[devicehostname] = getInterfaces(device)

      ###
      # Here we are connected, let collect neighbors
      ###
      new_neighbors = getNeighbors(device)
      neighborships[devicehostname] = new_neighbors

      
    '''
    NOW LETS PRINT OUR ALL NEIGHBORSHIPS FOR DEBUG
    '''
    pprint.pprint(neighborships)
    with open('neighborships.json', 'w') as outfile:
      json.dump(neighborships, outfile, sort_keys=True, indent=4)
      print("JSON printed into neighborships.json")  
      
    '''
    NOW LETS PRINT OUR ALL NEIGHBORSHIPS FOR DEBUG
    '''
    interfaces = filterNonActiveLinks(filterNonPhysicalLinks(interfaces))
    pprint.pprint(interfaces)
    with open('interfaces.json', 'w') as outfile:
      json.dump(interfaces, outfile, sort_keys=True, indent=4) 
      print("JSON printed into interfaces.json")
      
      
    '''
    GET INTERFACES WITHOUT NEIGHRBOR
    '''   
    print "====================================="
    print "no_neighbor_interfaces.json DICTIONARY "
    print "======================================"      
    interfacesWithoutNeighbor = filterLinksWithoutNeighbor(interfaces,neighborships)
    with open('no_neighbor_interfaces.json', 'w') as outfile:
      json.dump(interfacesWithoutNeighbor, outfile, sort_keys=True, indent=4) 
      print("JSON printed into no_neighbor_interfaces.json")    
    
    '''
    NOW LETS FORMAT THE DICTIONARY TO NEEDED D3 LIbary JSON
    '''
    print "================"
    print "NODES DICTIONARY"
    print "================"
    nodes_dict = getNodesFromNeighborships(neighborships)
    pprint.pprint(nodes_dict)
    
    print "================"
    print "LINKS DICTIONARY"
    print "================"    
    links_dict = getLinksFromNeighborships(neighborships)
    pprint.pprint(links_dict)
    

    print "=========================================="
    print "VISUALIZATION graph.json DICTIONARY MERGE"
    print "=========================================="
    visualization_dict = {'nodes':nodes_dict['nodes'],'links':links_dict['links']}
    
    with open('graph.json', 'w') as outfile:
        json.dump(visualization_dict, outfile, sort_keys=True, indent=4)
        print("")
        print("JSON printed into graph.json")
    
  # Bugfree exit at the end 
  quit(0)

Yes, I realize that this is an extremely long script, but instead of using all the script contents described in this article, I try very hard to use the comments in the code to describe its operation. The only real work you need to adjust for the laboratory is to change the configuration parameters in lines 15, 16 and 17. According to your laboratory naming convention, you may want to change the positive values in lines 33-36 Expression to match your logic.

Part 3. Check the output JSON

Therefore, after running the python script in part 2 in my lab topology, two JSON files are generated. The first is a simple description of the topology using a simple node (aka device) and link (aka interface) list. Here is graph.json:

{
    "nodes": [
        {
            "group": "2", 
            "id": "AR21-U12-ICB1"
        }, 
        {
            "group": "3", 
            "id": "usplnAGVPCLAB1003"
        }, 
        {
            "group": "1", 
            "id": "ng1-esx12"
        }, 
        {
            "group": "1", 
            "id": "ng1-esx11"
        }, 
        {
            "group": "2", 
            "id": "AR21-U12-ICB2"
        }, 
        {
            "group": "3", 
            "id": "usplnAGVPCLAB1004"
        }
    ],
    "links": [
        {
            "source": "AR21-U12-ICB1", 
            "target": "usplnAGVPCLAB1003", 
            "value": "40"
        }, 
        {
            "source": "AR21-U12-ICB1", 
            "target": "ng1-esx12", 
            "value": "10"
        }, 
        {
            "source": "AR21-U12-ICB1", 
            "target": "ng1-esx11", 
            "value": "10"
        }, 
        {
            "source": "AR21-U12-ICB2", 
            "target": "usplnAGVPCLAB1004", 
            "value": "40"
        }, 
        {
            "source": "AR21-U12-ICB2", 
            "target": "ng1-esx12", 
            "value": "10"
        }, 
        {
            "source": "AR21-U12-ICB2", 
            "target": "ng1-esx11", 
            "value": "10"
        }, 
        {
            "source": "AR21-U12-ICB2", 
            "target": "AR21-U12-ICB1", 
            "value": "10"
        }, 
        {
            "source": "AR21-U12-ICB2", 
            "target": "AR21-U12-ICB1", 
            "value": "10"
        }
    ]
}

OK, then there are two important JSON, called neighborhood. JSON and no_neighbor_interfaces.json, which store information about neighbors in a very similar way, but to save space, if you want to see their structure, see here.

The third part. Visualize graph.json using D3 JavaScript Library

My first inspiration came from viewing here Force orientation diagram of D3 library demo . It shows a simple but powerful code for visualizing the co-occurrence of characters in Victor Hugo's Les Mis é rables  ，  But you can also use input as a JSON structure for nodes and links. (to be honest, after I know this D3 demonstration, I generated this type of JSON in Python in the second part, so the python you see has been rewritten to create this type of JSON file structure).

The second and biggest change I made to the demonstration is that I need to use some form of hierarchy in my diagram to separate the core / aggregation / access layer and endpoint, so I hijacked the Y-axis gravity setting of D3 to create multiple artificial gravity lines. Look at lines 178-189 to see how I did it.

Again, as in the previous Python script in part 2, here is a complete JavaScript. I try to add some good comments to the code because there is not enough space to explain this line by line. But this code is simple.

<!DOCTYPE html>
<meta charset="utf-8">
<link rel='stylesheet' href='style.css' type='text/css' media='all' />

<body>
<table><tr><td>
  <svg width="100%" height="100%"></svg>
</td><td>
  <div class="infobox" id="infobox" >
      CLICK ON DEVICE FOR NEIGHBOR/INTERFACE INFORMATION
      <br>
      <a href="http://networkgeekstuff.com"><img src="img/logo.png" align="right" valign="top" width="150px" height="150px"></a>
  </div>
  <div class="infobox2" id="infobox2" >
  </div>  
</td></tr></table>
</body>
<script src="https://d3js.org/d3.v4.min.js"></script>
<script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"></script>
<script>

// =============================
// PRINTING DEVICE DETAILS TABLE
// =============================

// ====================
// READING OF JSON FILE 
// ====================
function readTextFile(file, callback) {
    var rawFile = new XMLHttpRequest();
    rawFile.overrideMimeType("application/json");
    rawFile.open("GET", file, true);
    rawFile.onreadystatechange = function() {
        if (rawFile.readyState === 4 && rawFile.status == "200") {
            callback(rawFile.responseText);
        }
    }
    rawFile.send(null);
}

function OnClickDetails(deviceid){
  //alert("devicedetails: " + deviceid);
  //usage:
  
  // #############################
  // # READING NEIGHBORS         #
  // #############################  
  readTextFile("python/neighborships.json", function(text){
      var data = JSON.parse(text);
      console.log(data); 
      console.log(deviceid);
      
      bFoundMatch = 0;  
      for (var key in data) {
        console.log("Key: " + key + " vs " + deviceid);

        if ((deviceid.localeCompare(key)) == 0){
          console.log("match!");
          bFoundMatch = 1;
          text = tableFromNeighbor(key,data);
          
          printToDivWithID("infobox","<h2><u>" + key + "</u></h2>" + text);          
        }
      }
      if (!(bFoundMatch)){
        warning_text = "<h4>The selected device id: ";
        warning_text+= deviceid;
        warning_text+= " is not in database!</h4>";
        warning_text+= "This is most probably as you clicked on edge node ";
        warning_text+= "that is not NETCONF data gathered, try clicking on its neighbors.";
        printToDivWithID("infobox",warning_text);
      }
  });  
  
  // ####################################
  // # READING NEIGHBOR-LESS INTERFACES #
  // ####################################
  readTextFile("python/no_neighbor_interfaces.json", function(text){
      var data = JSON.parse(text);
      console.log(data); 
      console.log(deviceid);
      
      bFoundMatch = 0;
      for (var key in data) {
        console.log("Key: " + key + " vs " + deviceid);

        if ((deviceid.localeCompare(key)) == 0){
          console.log("match!");
          bFoundMatch = 1;
          text = tableFromUnusedInterfaces(key,data);
          printToDivWithID("infobox2","<font color=\"red\">Enabled Interfaces without LLDP Neighbor:</font><br>" + text);          
        }
      }
      if (!(bFoundMatch)){
        printToDivWithID("infobox2","");
      }      
  });  
}

// ####################################
// # using input parameters returns 
// # HTML table with these inputs
// ####################################
function tableFromUnusedInterfaces(key,data){
  text = "<table class=\"infobox2\">";
  text+= "<thead><th><u><h4>LOCAL INT.</h4></u></th><th><u><h4>DESCRIPTION</h4></u></th><th><u><h4>Bandwith</h4></u></th>";
  text+= "</thead>";
  
  for (var neighbor in data[key]) {
    text+= "<tr>";
    
    console.log("local_intf:" + data[key][neighbor]['local_intf']);
    text+= "<td>" + data[key][neighbor]['local_intf'] + "</td>";
    console.log("description:" + data[key][neighbor]['description']);
    text+= "<td>" + data[key][neighbor]['description'] + "</td>";
    console.log("actual_bandwith:" + data[key][neighbor]['actual_bandwith']);
    text+= "<td>" + data[key][neighbor]['actual_bandwith'] + "</td>";
    
    text+= "</tr>";
  }  
  
  text+= "</table>";
  
  return text; 
}

// ####################################
// # using input parameters returns 
// # HTML table with these inputs
// ####################################
function tableFromNeighbor(key,data){
  text = "<table class=\"infobox\">";
  text+= "<thead><th><u><h4>LOCAL INT.</h4></u></th><th><u><h4>NEIGHBOR</h4></u></th><th><u><h4>NEIGHBOR'S INT</h4></u></th>";
  text+= "</thead>";
  
  for (var neighbor in data[key]) {
    text+= "<tr>";
    
    console.log("local_intf:" + data[key][neighbor]['local_intf']);
    text+= "<td>" + data[key][neighbor]['local_intf'] + "</td>";
    console.log("neighbor_intf:" + data[key][neighbor]['neighbor_intf']);
    text+= "<td>" + data[key][neighbor]['neighbor'] + "</td>";
    console.log("neighbor:" + data[key][neighbor]['neighbor']);
    text+= "<td>" + data[key][neighbor]['neighbor_intf'] + "</td>";
    
    text+= "</tr>";
  }  
  
  text+= "</table>";
  
  return text; 
}

// ####################################
// # replaces content of specified DIV
// ####################################
function printToDivWithID(id,text){
  div = document.getElementById(id);
  div.innerHTML = text;
}

// ########
// # MAIN #
// ########
var svg = d3.select("svg"),
    //width = +svg.attr("width"),
    //height = +svg.attr("height");
    width  = window.innerWidth || document.documentElement.clientWidth || document.body.clientWidth,
    height = window.innerHeight || document.documentElement.clientHeight || document.body.clientHeight;  
    
    d3.select("svg").attr("height",height)
    d3.select("svg").attr("width",width*0.7)  

var color = d3.scaleOrdinal(d3.schemeCategory20);

var simulation = d3.forceSimulation()
    .force("link", d3.forceLink().id(function(d) { return d.id; }).distance(100).strength(0.001))
    .force("charge", d3.forceManyBody().strength(-200).distanceMax(500).distanceMin(50))
		.force("x", d3.forceX(function(d){
			if(d.group === "1"){
				return 3*(width*0.7)/4
			} else if (d.group === "2"){
				return 2*(width*0.7)/4
			} else if (d.group === "3"){
				return 1*(width*0.7)/4                     
			} else {
				return 0*(width*0.7)/4 
			}
		 }).strength(1))
    .force("y", d3.forceY(height/2))
    .force("center", d3.forceCenter((width*0.7) / 2, height / 2))
    .force("collision", d3.forceCollide().radius(35));

// ######################################
// # Read graph.json and draw SVG graph #
// ######################################
d3.json("python/graph.json", function(error, graph) {
  if (error) throw error;

  var link = svg.append("g")
    .selectAll("line")
    .data(graph.links)
    .enter().append("line")
      .attr("stroke", function(d) { return color(parseInt(d.value)); })
      .attr("stroke-width", function(d) { return Math.sqrt(parseInt(d.value)); });

  var node = svg.append("g")
    .attr("class", "nodes") 
    .selectAll("a")
    .data(graph.nodes)
    .enter().append("a")
      .attr("target", '_blank')
      .attr("xlink:href",  function(d) { return (window.location.href + '?device=' + d.id) });

  node.on("click", function(d,i){  
      d3.event.preventDefault(); 
      d3.event.stopPropagation(); 
      OnClickDetails(d.id);
      } 
  ); 

  node.call(d3.drag()
          .on("start", dragstarted)
          .on("drag", dragged)
          .on("end", dragended));  
          
  node.append("image")
      .attr("xlink:href", function(d) { return ("img/group" + d.group + ".png"); })
      .attr("width", 32)
      .attr("height", 32)
      .attr("x", - 16)
      .attr("y", - 16)      
      .attr("fill", function(d) { return color(d.group); }); 
      
  node.append("text")
      .attr("font-size", "0.8em") 
      .attr("dx", 12)
      .attr("dy", ".35em")
      .attr("x", +8)
      .text(function(d) { return d.id });                

  node.append("title")
      .text(function(d) { return d.id; });

  simulation
      .nodes(graph.nodes)
      .on("tick", ticked);

  simulation.force("link")
      .links(graph.links);

  function ticked() {
    link
        .attr("x1", function(d) { return d.source.x; })
        .attr("y1", function(d) { return d.source.y; })
        .attr("x2", function(d) { return d.target.x; })
        .attr("y2", function(d) { return d.target.y; });

    node
        .attr("transform", function(d) { return "translate(" + d.x + "," + d.y + ")"});        
  }
});

function dragstarted(d) {
  if (!d3.event.active) simulation.alphaTarget(0.3).restart();
  d.fx = d.x;
  d.fy = d.y;
}

function dragged(d) {
  d.fx = d3.event.x;
  d.fy = d3.event.y;
}

function dragended(d) {
  if (!d3.event.active) simulation.alphaTarget(0);
  d.fx = null;
  d.fy = null;
}
</script>

Part IV. Final visualization of my lab

When you combine the JSON file generated by the python script in part 2 with the D3 visualization in part 3, your result on a subminiature topology like my LAB will be as follows (now use the screenshot):

Boring, right? Click here to view live JavaScript animation.

Yes, but this is not limited to my small laboratory. I actually try to run it with a larger number of devices in the laboratory. That's the result. Again Click here to view Bigger Live demonstration.

Example of python/JavaScript visualization using the display on a larger LAB

Click here for a larger live JavaScript animation.

generalization

Well, here I share with you my quick experiment / example to illustrate how to use NETCONF+python+PyHPEcw7's first quick map to quickly visualize the network topology, and then use Javascript+D3 library to visualize it to create an interactive self-organizing map. After clicking it, the topology of each device and some small information pop-up windows will pop up. You can Download the entire project as a package here.

Follow up September 2019

In subsequent articles of this article, I updated this visualization example to include traffic pattern data and provide a better visual theme for it. please Focus here Or click on the picture below. In addition, in this update, I publicly shared the code through github and used SNMP as the data source, which is a more popular and requested function.

--- Peter Havrila , published in
December 12, 2017