What is the Challenge?
Successful enterprise data processing systems have a tendency to grow out of their well managed walled-gardens. This happens because they need additional resources and tools found outside the garden, e.g. offered by a Cloud provider, or because they start to connect into other enterprise systems. Traditional metadata catalogs do not scale well to handle this organic growth, depending on static and predefined processes to manage and curate data within some well defined perimeter. This makes enterprise data management an intractable problem as the data is constantly been moved and processed between these heterogeneous systems by a large number of independently managed tools.
Significant value can be enabled by gathering, linking and enriching metadata at enterprise scale allowing cost reduction through, for example identifying cold data, removing unnecessary data duplication or better scheduling of workflows. New value added services can be introduced with the guarantee that sensitive data is being used in the way it is intended and being properly managed and protected.
This is analogous to the ways in which enterprises have unlocked hidden value in their data through big data processing. The processing of heterogeneous metadata at scale is in fact a type of big data problem with the same classic 5 Vs of: velocity, value, variety, volume and veracity.
Automated discovery and metadata collection
Extensive heterogeneous metadata sources
Metadata enriched and linked creating Data Map
Event-based notification of key changes
Supporting a wide range of data management applications
Pathfinder is an event-based system that dynamically collects metadata from data catalogs, data stores (databases, file systems) and systems that transform and process data. The metadata is linked, enriched and stored in data maps tailored for data-management applications.