Model Persistence For Network Inventory (Strawman)

Overview

In effort to meet the needs of the ZipTie community several different persistence strategies are currently being evaluated. This document summarizes the current set of options under consideration for model persistence.

Requirements

Persistence is an implementation detail. The goal for persistence is to enable ease creation of tools and applications for ZipTie. In addition as an Inventory Framework, ZipTie should enable basic reporting capabilities through it's persist.

Inventory Based Tools

ZipTie as an inventory tools framework is built on the based on the principle that by aggregating the entire network in a searchable inventory, users can create powerful tools to automate network management tasks. Today networks are not easily automatable for two key reasons:

• Network devices to do not provide uniform management interfaces that lend themselves to automation.
• Network management data is irregular (not-normalized) and is scattered in a variety of stores throughout the physcial network devices.

Use Cases

Nicolas (Network Scripter Persona) want to create tool which reboots devices, which haven't been rebooted in over 6 months. In order to create this tool Nicolas uses the ZipTie repository to find all the devices today -last reboot time is greater the 6 months.

1. It must be easy to access ZipTie data from script, framework or ZipTie application (Java). Today leveraging network configuration data is difficult, due to the fact that scipts must go to the device to access the data and the data is not uniformly formatted.
2. The ZipTie persistence strategy must normalize the data enable automation.
3. The ZipTie persitence must be able to easily persist the ZipTie Network Resource Model.
4. The ZipTie persistence must scale to large networks (up to 10,000 devices).

Sysco (EMS Developer) has chosen ZipTie as the network inventory technology for her EMS product. Sysco wants to leverage the exinsting tools that ZipTie provides for her customers and add additional features based on the specific features, which differentiate her company's equipment. Sysco like most good developers wants to develop the most amount of functionality with the least amount of effort. Sysco company's has chosed Java for their EMS and Sysco's development team has used various open source components for thier EMS product. Sysco's company makes hardware and does not believe that their EMS software provides any value add for their hardware. Sysco's company believes EMS development is necessary evil.

Sysco manager is interested in advanced mangement concepts like ITIL, Compliance and CMDB's and asks Sysco if there is any way to include some of that stuff in their box.

1. The ZipTie persistence must be able to be extended easily as the NRM is extended
2. The ZipTie persistence must enable basic inventory reporting capability: Show me all my , filtered by attribute, where X is a model element. Example: Show me all my devices, Show me all my chassis, Show me all my interfaces

Sysco (EMS Developer) and her development manager chose ZipTie to use inside their EMS to provide inventory and configuration capability. Sysco's company deployed a ZipTie based EMS for their Goldberg 9000 network hardware. Sysco's product manager has just anounced that Sysco's company will be releasing the Goldberg 9001, with better support for application tuning parameters. Sysco's company has patented application traffic shaping technology know as AppGiblets and Sysco needs to add support for AppGiblets technology in EMS built with ZipTie technology.

1 The ZipTie persistence must be able to accomodate migration of one schema version to a new schema version based on model extension.

Proposed Architecture

The following diagram shows a proposed architecture for meeting the following requirements:

There two key components to the proposed ZipTie persistence architecture: snap shot RDBMS and the configuration text version history. With M2, ZipTie supports configuration text version history on the basis of RCS. This allows the history of configuration to be captured and stored as source files, within the RCS system. In order to provide mode capabilities for tools and to enable basic inventory reporting, we propose to enable a single snapshot database (JDBC compliant) of the model stored as tables. By doing so, a ZipTie script writer can use SQL to query the entire model database to create functionality. A broad array of inventory based tools can be created, once the ZipTie framework includes support for an RDMBS. It is the goal of this hybrid architecture to leverage both the flexibility of file based storage and the features of an RDMBS in a single persist. The inclusion of both these technology provides choice for ZipTie developers.

Key Challenges

By choosing support for an RDBMS, ZipTie is confronted with several key challenges:

• How do provide a model flexible enough to support all devices and still provide enough common capabilities to get tool leverage
• Tactically, if ZipTie has a flexible model, how does that model get mapped into a set of RDBMS tables, which enable basic reproting

Mapping Technologies

The folllowing is a survey of mapping technologies, which have been considered for mapping the NRM to a RDBMS in ZipTie

Hand Mapping

The most basic approach for the ZipTie framework is leave the RDBMS persist to the Adapter developer. Once the NRM is defined for the Adapter, there would be a corresponding code module executed by the ZipTie framework to persist the model.

Pro's

• Less stuff in the ZipTie framework
• Conceptually simple: Your code here!

Con's

• Potential for conflict within the ZipTie schema. How are collisions arbitrated?
• Tools leverage? How can queries "across" an inventory of devices be written, when entities are "non standard". Some basic schema rules/conventions must be inplace to enable this, but they would have adhered to by the ZipTie developer.

Fixed Mapping

ZipTie provides a set of pre-defined tables for common and core types. ZipTie moderates the schema changes. Extensions are 'outside' the fixed schema and left to the developer creating the extension. The schema migrates with each rev of ZipTie.

Pro's

• Mapping is well known by all tools
• Conceptually simple: Your code here!

Con's

• Tools leverage? How can queries "across" an inventory of devices be written, when entities are "non standard" extensions. Some basic schema rules/conventions must be inplace to enable this, but they would have adhered to by the ZipTie developer.

Automated Mapping (Metaschema Approach)

From the DMTF : The Meta Schema is a formal definition of the model. It defines the terms used to express the model and its usage and semantics.

By developing a metaschema, a general mapping technology can be used in order to persist the NRM is an RDBMS. The metaschema defines the mapping rules so that as the schema is extended, a RDMBS schema is producted out by the mapping technology, which meets ZipTie's general requirements. The metaschema approach constrains the NRM in specific ways in order to enable interoperability across the schema.

Examples of this interoperability include:

• Identifying attributes: By defining id's within the schema, RDBMS mapping can leverage ID/KEY constraints
• Star schema for basic reporting. A mapping technology can produce a natural star schema from the NRM hierarchy of types.
• Mapping derived entities. A mapping technology can allow existing types to be encoded by the RDBMS to enable easy querying and reporting

Pro's

• Potentially eliminates the need to write SQL and reporting schema by Adapter developer: Say goodbye to a bunch of typing.
• Provides automation for extension and schema migration

Con's

• Abstraction of database specifics can create bad database schema if rules are not followede