pyegeria 5.3.9.9.3__py3-none-any.whl → 5.3.9.9.5__py3-none-any.whl

md_processing/dr_egeria_inbox/archive/dr_egeria_intro_part1.md

@@ -0,0 +1,254 @@
+# Introduction to Dr.Egeria - an Egeria Markdown Processor  
+      
+      
+A constant challenge in managing information is gathering enough metadata about the information to  
+allow us to manage it. A common approach is to build fancy graphical user interfaces hoping that they  
+will be attractive enough and easy enough to use that people will do so.  
+      
+ Unfortunately, however, this ignores the fundamental fact that to use one of these nice GUI  
+ applications, you have to step away from the tools and processes that you were in the midst of performing.  
+ You have to leave your world and enter a new, often less familiar one.  
+      
+Dr.Egeria, is an experiment in turning this around. Its not that fancy graphical user  
+interfaces don't have a role - but rather, to look at what we can do to support the  
+tools and approaches people already use. And maybe even make their day job a little  
+easier and a little more enjoyable.  
+      
+So this is what we are exploring with Dr.Egeria. An Egeria Markdown language that allows  
+users to intermix requests to Egeria with other text through the use of standard Markdown. The markdown text  
+that we process can be in standard markdown (.md) files, in Jupyter notebooks, and perhaps other file formats.   
+      
+This markdown file is an example. You will see that we intersperse normal narrative text (such as this)  
+with Commands to Egeria. We introduce a specific vocabulary to make Egeria requests.  
+      
+In the example below we will create a new Egeria glossary to hold definitions related to Dr.Egeria.   
+We will then show how we can process this file which will record the information into Egeria and create a new  
+output file that acts as both a receipt showing what was processed as well as a starting point for making updates.  
+      
+So here we go! First lets define a new Glossary::  
+      
+___  
+      
+# Create Glossary  
+      
+## Glossary Name  
+      
+Egeria-Markdown  
+      
+## Language  
+      
+English  
+      
+## Description  
+      
+Glossary to describe the vocabulary of Dr.Egeria - an Egeria Markdown language to support the exchange of metadata in a Markdown form.  
+Dr.Egeria allows users to input metadata using any text entry system that supports the entry of standard Markdown  
+notation and through post-processing  
+commands, validates the Egeria content and allows the requests to be sent to Egeria.   
+      
+## Usage  
+      
+1. (optional) load an example or template for the type of object from Egeria.  
+2. Create a new document (perhaps from a template) and edit it, adding in the content with the Dr.Egeria controlled Markdown language.  
+3. Process the document to validate and display it before you submit it, Validation may annotate your document with recommendations and potential issues.  
+4. Submit the document to Egeria using the Dr.Egeria commands.  
+
+      
+___  
+      
+      
+      
+      
+# First Walk-Through  
+The block of markdown above is a request to create a new Egeria Glossary called `Egeria-Markdown`. Let's briefly walk  
+through. The command starts when we see `# Create Glossary`. This is a known phrase in Dr.Egeria. When we see this   
+phrase we recognize that this is an Egeria markdown request block. The request block ends if we encounter another `#` or  
+`___`, or run out of text. Within this request block we note some **attributes** that begin with a `## `. The first that  we encounter is `## Glossary Name`. Not all attributes need to be filled in. Later, we'll process this file and demonstrate  how to tell - but first, lets look at the attributes shown:  
+      
+* `## Glossary Name` - this is the display name of the glossary. In this case the name is `Egeria-Markdown` As you can see, the value of the attribute is the plain text that follows it.  
+* `## Language` - what language will the terms of the glossary be in (yes there are ways to have mixed language but  Dr.Egeria strives to be as simple as possible).  
+* `## Description` - a description of the glossary and its purpose.  
+* `## Usage` - how the glossary is meant to be used and by whom.  
+* `## <Qualified Name>` - every element in Egeria must have a unique qualified name that we use to distinguish it from all other elements. The qualified name is meant to be understandable by humans, although it may follow formatting conventions. This attributes is in angle brackets because at this point we can't fill it in - 
+we are just in the midst of creating the glossary. A qualified name will be created for us as part of the glossary creation. We'll see a little later how we get that.  
+ * `## <GUID>` - same story as qualified name except that this is meant for automation and not people.  
+      
+And that's it. That's all we need to do to specify the creation of a new glossary (well - mostly - we'll reveal a few   
+more details a bit later).  
+      
+## Great! That was easy!  
+      
+We now have a nice, clean, new...and empty...glossary - guess we better start filling it. Lets create a couple of terms.  
+      
+___  
+      
+# Create Term  
+      
+## Term Name  
+      
+Command  
+      
+## In Glossary  
+      
+Glossary::Egeria-Markdown  
+      
+      
+## Summary  
+      
+Commands are how a user of the Freddie markdown language requests an action.  
+      
+## Description  
+      
+Commands are how a user can request Egeria to take an action such as Create or Update an Egeria element. Freddie  
+provides a limited (but growing) set of commands. Freddie commands align with the pyegeria 'hey-egeria' 
+command line interface.  
+      
+## Abbreviation  
+      
+## Examples  
+      
+    Create Glossary or  
+    Update Glossary or  
+    Create Term or  
+    Update Term  
+      
+## Usage  
+      
+    Commands are used in the Freddie Egeria markdown language.  
+      
+## Version  
+      
+   0.1   
+      
+## Status  
+      
+      
+## <Qualified Name>  
+      
+## <GUID>  
+      
+___  
+      
+# Create Term  
+      
+## In Glossary  
+      
+    Glossary::Egeria-Markdown  
+      
+## Term Name  
+      
+    Source  
+      
+## Summary  
+      
+    Source of the markdown content.  
+      
+## Description  
+      
+    Source of the markdown content - could be jupter or plain markdown file.  
+      
+## Abbreviation  
+      
+## Examples  
+      
+## Usage  
+      
+## Version  
+      
+    0.1  
+      
+## Status  
+      
+    DRAFT  
+      
+      
+___  
+      
+# Create Term  
+      
+## In Glossary  
+      
+    Glossary::Egeria-Markdown  
+      
+## Term Name  
+      
+    Directive  
+      
+## Summary  
+      
+    A directive defines how the command is to be processed.  
+      
+## Description  
+      
+    Directives are one of:  
+      
+* display - just display what we've found  
+* validate - check the validity of the requested action  
+* process - process the requested action  
+      
+      
+## Version  
+      
+    0.1  
+      
+## Status  
+      
+    DRAFT  
+      
+___  
+      
+# Some terms specified - Now what?  
+      
+Ok - we've now defined a glossary and three terms to go into the glossary. A few observations.  
+      
+* There is a degree of freedom in writing the definitions. The attributes aren't always in the same  
+order and optional attributes don't need to be specified at all. We try to make things as easy as possible to use.  
+* We can run a definition file through a validation process to check our proposed definition and provide feedback.  
+* When we process a definition we will use the same validation process before trying to update Egeria   
+with the requested definitions - requests may get rejected or altered - this will be consistent with the feedback we   
+provide during validation.  
+      
+Here is what we'll do next.  
+      
+## Next  
+      
+ >Note: This is changing - so will be somewhat abstract  
+      
+We will run a small program called `dr.egeria.py` to operate on this markdown file. When we run this program we tell it not just the name of the file to process but also provide a directive on what to do. Currently we have the   
+choice of:  
+      
+1. Display - just parse the file, breaking it down into request blocks, and display what we find  
+2. Validate - parse the file and validate if the commands can be processed - showing information about what we observe.  
+3. Process - parse the request blocks and execute the commands - and produce a new output file to simplify further processing.  
+      
+      
+# Great --> let's give it a try!  
+      
+Ok - its processed the file and generated output to the console that shows us what it has done.  
+We also now have a new file in the designated outbox (specified by an Environment Variable).  
+If we review that file, we see that it has similar content to this original file except that  
+the **Create** statements have been replaced with **Update** statements and  
+attributes such as **Qualified **Name** and **GUID** now contain the known values.  
+      
+This means that if we want to make changes to the definitions that we have   
+created, all we need to do is to make changes to the updatable attributes in this  
+new document and resubmit it - pretty simple.  
+
+Here is a diagram of this process from the user perspective:
+
+```mermaid
+flowchart TD
+    A[Text Editor or Jupyter Notebook] --> B(dr.egeria content)
+    B --> C{dr.egeria command}
+    C -->|display| D[console output]
+    C -->|validate| E[console validation output and file]
+    C -->|process| F[console output and processed output file]
+    F-->|Additional Updates|A
+    E-->|Additional Updates|A
+ ``` 
+      
+In the next section we'll see how we can update and extend what we have done by creating  
+some glossary categories and then assigning categories to the terms.  
+To do this we will copy the output document that we just created and call the   
+copy dr_egeria_intro_part2.md. The text of the document has also been updated  
+to reflect the purpose. Let's open that now!

md_processing/dr_egeria_inbox/archive/dr_egeria_intro_part2.md

@@ -0,0 +1,298 @@
+# Demonstrating updates and additions  
+      
+This document was derived from the processing of the file dr_egeria_intro_part1.md; All the Dr.Egeria statements have been 
+preserved (including the generated attributes such as the unique GUIDs), and the explanatory text has been replaced.
+
+In this document we will add glossary categories, update existing terms to categorize them, and add some new terms that
+further explain dr.egeria.  This document is being written off-line (actually on a plane) - demonstrating one of the 
+intriguing features of dr.egeria - because its just text, it can be edited anywhere and anytime that there is a text 
+editor. Dr.Egeria files can be emailed, sent via text message, slack, or maintained in a `git` repository. 
+It's just text with markdown annotations.
+
+Ok, let's get started. First, we have the `Update Glossary` command below. There is nothing new we need to add at this
+point so we can just leave it as-is. When this document is processed it will apply updates but if there are none, 
+It doesn't matter.
+
+However, now is a good time to start to more formally specify the attributes available to `Create` or `Update` glossary
+commands:
+
+
+| Attribute Name | Input Required? | Read Only | Generated/Default? | Unique? | Notes                                                                                                    |
+|:---------------|:----------------|-----------|:-------------------|:--------|:---------------------------------------------------------------------------------------------------------|
+| Glossary Name  | Yes             | No        | No                 | No      | A display name (informal name).                                                                          |
+| Language       | No              | No        | No                 | No      | The primary language for the glossary.                                                                   |
+| Description    | No              | No        | No                 | No      | A textual description of this glossary.                                                                  |
+| Usage          | No              | No        | No                 | No      | How the glossary is meant to be used, and by whom.                                                       |
+| Qualified Name | Maybe           | No        | Yes                | Yes     | The qualified name can either be provided by the user or generated. If generated, a pattern is followed. |
+| GUID           | No              | Yes       | Yes                | Yes     | GUIDs are always generated by Egeria. They are meant for automation, not people.                         |
+
+The table above shows us what attributes must be provided and which are optional. Once an object is created, and a qualified name is generated, it is good practice to use it as there can 
+be many objects with the same name, and this is a common way to disambiguate them. If you specify an object name (e.g. Glossary Name) and that name already exists, Dr.Egeria will report an error
+and suggest that you provide a **Qualified Name** as well. You will also note that the GUID is generated by Egeria and is read-only. It is possible that some commands
+may require a GUID to be specified, but in general we will use the **Qualified Name** to identify objects. 
+
+> Comment: In the Egeria and pyegeria APIs, Glossary Name is actually referenced as a **Display Name**. It is perfectly fine for multiple objects to have the same display name.
+> However, it is required that even though the may share the same display name, they must have different qualified names and GUIDs. So if you provide a Glossary Name that already exists,
+> you will get an error message that suggests that you provide a Qualified Name in addition to the Glossary Name.
+
+___ 
+
+# Glossary Categories
+
+Sometimes it can be useful to provide more structure to the glossary. The way to do this is through categories. 
+In Egeria, a category can have a single parent category and multiple child categories. A term can be assigned to 
+multiple categories. When we have a large number of terms, a category structure can be particularly helpful in finding the
+terms that are most relevant to a particular topic. Using categories is optional.
+
+Ok, now let's create a couple of glossary categories. They will be:
+
+* **Writing Dr.Egeria Markdown** - where we describe elements of the Dr.Egeria language as terms within the category.
+* **Processing Dr.Egeria Markdown** - where we describe the commands for processing Dr.Egeria.
+ 
+
+Glossary categories have the following attributes:
+
+
+| Attribute Name  | Input Required? | Read Only | Generated/Default? | Unique? | Notes                                                                                                    |
+|:----------------|:----------------|-----------|:-------------------|:--------|:---------------------------------------------------------------------------------------------------------|
+| Category Name   | Yes             | No        | No                 | No      | A display name (informal name).                                                                          |
+| Owning Glossary | Yes             | No        | No                 | Yes     | This is the qualified name of the glossary that owns this category.                                      |
+| Description     | No              | No        | No                 | No      | A textual description of this category                                                                   |
+| Qualified Name  | Maybe           | No        | Yes                | Yes     | The qualified name can either be provided by the user or generated. If generated, a pattern is followed. |
+| GUID            | No              | Yes       | Yes                | Yes     | GUIDs are always generated by Egeria. They are meant for automation, not people.                         |
+
+> Note: Qualified Names can either be user specified or generated. If generated the following the form:
+`{local-qualifier}::{type}::{display name}::{version}`. Local-Qualifier is an optional string that can be useful to both disambiguate similar names and to add some local context. Local qualifiers could be set to organization names, functions, business context, etc. The settings for a local qualifier is set either by setting the environment variable `EGERIA_LOCAL_QUALIFIER` or by passing in a parameter when executing one of the Dr.Egeria enabled commands. This could also be set for a team by an Egeria administrator.
+
+Ok, here we go:
+
+___
+
+# Create Category  
+  
+## Category Name  
+  
+Writing Dr.Egeria Markdown  
+  
+## Owning Glossary  
+  
+Glossary::Egeria-Markdown  
+  
+## Description  
+  
+These terms describe the elements of the Dr.Egeria Markdown language and how to use them.   
+
+
+___
+
+# Create Category  
+  
+## Category Name  
+  
+Processing Dr.Egeria Markdown  
+  
+## In Glossary  
+  
+Glossary::Egeria-Markdown  
+  
+## Description  
+  
+These terms describe commands to process Dr.Egeria Markdown.  
+   
+
+___  
+
+> Note: If you look at the examples above, you will notice that sometimes we specify **In Glossary** and sometimes we specify **Owning Glossary**. 
+> There is no difference. We will have preferred names that we will use when we generate a Dr.Egeria markdown file, but we try
+> to be flexible and allow for common name variations. You will see how we document this as we proceed.
+  
+Now, let's add categories to the terms.  Let's review the attributes of a term:
+
+
+| Attribute Name  | Input Required? | Read Only | Generated/Default? | Unique? | Notes                                                                                                                |
+|:----------------|:----------------|:----------|:-------------------|:--------|:---------------------------------------------------------------------------------------------------------------------|
+| Term Name       | Yes             | No        | No                 | No      | A display name (informal name).                                                                                      |
+| Owning Glossary | Yes             | No        | No                 | Yes     | This is the qualified name of the glossary that owns this term.                                                      |
+| Categories      | No              | No        | No                 | Yes     | This is the name of the category. Multiple categories can be assigned, separated by a `,` or line.                   | 
+| Description     | No              | No        | No                 | No      | A textual description of this term                                                                                   |
+| Qualified Name  | No              | Yes       | No                 | Yes     | The qualified name can either be provided by the user or generated. If generated, a pattern is followed.             |
+| GUID            | No              | Yes       | Yes                | Yes     | GUIDs are always generated by Egeria. They are meant for automation, not people.                                     |
+
+> When we provide a category name in the `Categories` attribute, We can use either the display name form (Category Name) or the qualified name form. If we find that the display name is not unique, 
+> you will need to provide the qualified name. Its safer to use the qualified name, but a little less readable.
+
+___  
+
+# Update Term
+
+## Term Name 
+
+Command
+
+## Summary
+Commands are how a user of the Dr.Egeria markdown language requests an action.
+
+## In Glossary
+Glossary::Egeria-Markdown
+
+## Categories
+
+Category::Writing Dr.Egeria Markdown, Category::Processing Dr.Egeria Markdown
+
+## Status
+ACTIVE
+
+## Description
+Commands are how a user can request Egeria to take an action such as Create or Update an Egeria element. Freddie
+provides
+a limited (but growing) set of commands. Dr.Egeria commands align with the pyegeria 'hey-egeria' command line interface.
+
+## Examples
+Create Glossary or
+Update Glossary or
+Create Term or
+Update Term
+
+## Usage
+Commands are used in the Dr.Egeria markdown language.
+
+## Published Version
+
+0.2
+
+## Qualified Name
+
+
+
+___
+
+# Update Term
+
+## Term Name 
+
+Source
+
+## Summary
+Source of the markdown content.
+
+## In Glossary
+Glossary::Egeria-Markdown
+
+## Categories
+
+Processing Dr.Egeria Markdown 
+
+## Status
+ACTIVE
+
+## Description
+Source of the markdown content - could be jupyter or plain markdown file.
+
+## Examples
+
+## Usage
+
+
+## Published Version
+
+0.2
+
+## Qualified Name
+Term::Source::0.1
+
+
+___
+
+# Update Term
+  
+## In Glossary  
+  
+Glossary::Egeria-Markdown  
+  
+## Term Name  
+  
+Directive  
+  
+## Categories  
+  
+Processing Dr.Egeria Markdown  
+  
+## Summary  
+  
+A directive defines how the command is to be processed.  
+  
+## Description  
+  
+Directives are one of:  
+  
+* display - just display what we've found  
+* validate - check the validity of the requested action  
+* process - process the requested action  
+ 
+  
+  
+## Version  
+  
+0.2 
+  
+## Status  
+  
+DRAFT  
+  
+## Qualified Name  
+Term::Directive::0.1  
+
+  
+# Inspecting the Glossary
+
+Now that we have created a glossary, categories, and terms we can use some new commands to show us what we have done.
+We will start with the `List Glossaries` command. This command will list all the glossaries that are available to us.
+
+___
+
+# List Glossaries
+
+___
+
+This will return a markdown table of all known glossaries based on the defaults set for the optional attributes. Here 
+is a more detailed specification of the attributes:
+
+| Attribute Name | Input Required? | Read Only? | Generated/Default?               | Unique? | Notes                                     |
+|----------------|-----------------|------------|----------------------------------|---------|-------------------------------------------|
+| Search String  | No              | No         | default is All glossaries        | No      |                                           |
+| Output Format  | No              | No         | default is Markdown List (table) | No      | options are: LIST, DICT, MD, FORM, REPORT |
+
+Lets describe the output formats a bit further:
+
+* LIST - This is the default format. It returns a markdown table of the glossaries.
+* DICT - This returns a python dictionary (or JSON representation) of the glossaries. 
+* MD - This returns a markdown document of the glossaries.
+* FORM - This returns a Dr.Egeria markdown form designed to be used as a starting point for updating the glossary definitions.
+* REPORT - This returns a markdown document of the glossaries that is designed to be more readable and perhaps suitable to be used in a report.
+
+Going further, we can issue similar commands to list categories and terms:
+
+The attributes for the `List Categories` command are the same as the `List Glossaries` command:
+Attributes for the `List Terms` command are:
+
+
+| Attribute Name | Input Required? | Read Only? | Generated/Default?               | Unique? | Notes                                     |
+|----------------|-----------------|------------|----------------------------------|---------|-------------------------------------------|
+| Glossary Name  | No              | No         | Default is All glossaries        | No      |                                           |
+| Search String  | No              | No         | default is All terms             | No      |                                           |
+| Output Format  | No              | No         | default is Markdown List (table) | No      | options are: LIST, DICT, MD, FORM, REPORT |
+
+
+
+___
+
+# List Categories
+## Output Format
+REPORT
+
+___
+# List Terms
+## Output Format
+DICT
+
+