Link Search Menu Expand Document
Jamie Sullivan's Blog

1.0 Introduction - Network and Infra Monitoring with Grafana, Django and Python

In this series, we use REST APIs, Python, Django Webframework, PostgresSQL and Grafana to demonstrate building a cross-domain visualisations Dashboard for Cisco network and Infrastructure.

1.2 Lab Overview

alt text

1.3 Overview and Functional Components

1.3.1 Django

  • Extensible and Flexible: Django allow’s additional functionality to be added (e.g. user input, data presentation) as required
  • Database Abstraction: Django’s SQL abstraction API means we can work with databases in pythonic way. Database tables are defined in static, Model Files and then pushed/ commited to the database… one less language (SQL) to learn.
  • Familiarity: Access to tutorials and documentation

1.3.1.1 Django Resources

1.3.2 Postgres SQL

  • Tool availability: Tools such as PgAdmin reduce complexity and knowledge needed to interact with and validate databases
  • Integration: SQLLite is the default database deployed with Django, Postgres is also fully supported by both the Django models/database API and Grafana for visualisations.
  • Scalability
  • Simple requirements: For a mock lab and self learning, Postgres SQL has more capability than I need. For production environments you’ll want to analyse requirements specific to your usecase and understand the pros and cons of timeseries databases such as InfluxDB, prometheus and Logstash for event/syslog aggregation.

1.3.2.1 PostgresSQL Resources

1.3.3 Python

Python Requests module used to make REST API calls to various controllers, ingest replies in JSON encoded format. From the returned JSON / Python Dictionary, use the Django database abstraction API to write returned JSON into a table hosted on a Postgres SQL database.

The django Webframework package is also written in Python.

We’ll step through defining our django models, ingesting REST-API output from controller and cloud APIs into the django models, verifying table entries using PgAdmin and ultimately creating Dashboards such as below.

At the end of this series, we’ll created basic dashboards and visualisations such as below for:

  • Meraki Dashboard
  • Cisco SDWAN (vManage)
  • Cisco ACI
  • DCNM VXLAN and EVPN
  • Cisco DNAC
  • Openvuln API for PSIRT reporting
Meraki Devices:

alt text

DNAC Devices:

alt text

Software Defined WAN Use Case

alt text

Openvuln API Dashboard

alt text

1.4 Initial Setup

1.4.1 Create and activate a virtualenvironment and initialise an empty git repository

zsh-shell Documents % mkdir grafana-cisco-demo             
zsh-shell Documents % cd grafana-cisco-demo
zsh-shell grafana-cisco-demo % virtualenv venv
zsh-shell grafana-cisco-demo % source venv/bin/activate
(venv) zsh-shell grafana-cisco-demo % git init
(venv) zsh-shell grafana-cisco-demo % nano .gitignore
^^ Add the venv/ directory to above .gitignore file

1.4.2 Install django

(venv) zsh-shell grafana-cisco-demo % python -m pip install django
(venv) zsh-shell grafana-cisco-demo % python -m django --version
3.2
(venv) zsh-shell grafana-cisco-demo % cd cisco_grafana
(venv) zsh-shell cisco_grafana % python manage.py runserver
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

1.4.3 Install postgresql - or supported database of your choice

For installation options see - Postgres Installation

Once you’ve validated postgres’s installed, edit your django settings file, so that django points to the postgres database.

# Database
# https://docs.djangoproject.com/en/3.1/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'postgres',
        'USER': 'postgres',
        'PASSWORD': 'postgres',
        'HOST': '127.0.0.1',
        'PORT': '5432',
    }
}

Install python postgres sql package

(venv) zsh-shell cisco_grafana % pip install psycopg2

1.5 Wrap-up

We’ll begin working with controller APIs, making requests, parsing responses, ingesting into our database and generating visualisations in the next update.

In the meantime - checkout DevNet if you haven’t already. We’ll re-use a lot of the code and examples shared in the DevNet Learning Tracks and the sandbox environments.

Continue to Part 2 [meraki] : building a multidomain dashboard with cisco apis and grafana