All posts by kozzeluc

Installations All Around the World

I just wanted to post an updated image showing you WHERE the Eclipse Schema Diagram Editor was installed during the last half year.  I must say I’m both impressed and surprised (ad of course pleased) to see that my software has been installed all around the world.  I’m curious whether you installed the diagram editor because you’re using IDMS or whether you’re in the process of writing a graphical editor for Eclipse with the Graphical Editing Framework (GEF) and just wanted to look at an example… please let me know by commenting on this post ! 😉

installs

Advertisements

Old Update Site Removed

The old update site (http://users.skynet.be/dmlj) has already been removed because with 30 installs/updates last month, I think most of you can find their way to the new update site at Bintray (https://dl.bintray.com/kozzeluc/dmlj/latest).

The following illustration (taken from Bintray) shows where the Eclipse CA IDMS™/DB Schema Diagram Editor has been installed or updated last month :

installs_20170429_122000

The archived update site was downloaded in Germany, but the above map is for the live update site only (that’s why Germany is white).  If your country is white but you would like to see it colored, make sure to install or update the diagram editor somewhere this May 😉

Update Site has moved to Bintray

Because the space I have available from my internet provider is only 50Mb and because we started experiencing problems when installing due to (I guess) a firewall issue, I needed to look for an alternative for the diagram editor’s Eclipse update site.  Bintray offers a lot more space, so that from now on I can publish each new version while retaining the previous ones (the actual update site however only contains the latest version).

Apart from installing through HTTPS instead of HTTP, you shouldn’t notice too many differences when installing the product for the first time via the Eclipse Marketplace Client, nothing has changed in that respect (I hope you don’t have too much trouble because of HTTPS, you might need to talk to your firewall guys or download the archive with your browser).

If you already have installed the diagram editor and you would like to continue to receive updates, you should change the update site in the Eclipse preferences.  Look for the update site definition and change the URL from http://users.skynet.be/luc_h/dmlj to https://dl.bintray.com/kozzeluc/dmlj/latest :

new_update_site

If you make this modification and run ‘Help/Check for Updates’, you can update to the new version 3.1.0.  The only new feature is the ‘Copy to Clipboard‘ menu item, which lets you copy your entire diagram to your clipboard, allowing you to paste it in the graphics program of your choice (Paint, …).  I think this can be of help when cutting up your diagram into several pages (because printing large diagrams is still problematic in the diagram editor).

For now, version 3.0.2 will stay for some time at the old update site URL, but I won’t put any new version there.  So you really should change your update site definition.

The installation manual has been updated.  You can now download the update site archive from Bintray and stash it on your local file system; this wasn’t possible in the past.

Managing Diagram Versions with GIT

You have several options for versioning your IDMS diagrams, but I think Git is one of the most interesting today, because it allows you to do versioning WITHOUT the need for a server AND it is free 😉 On top of that, Eclipse comes with a fairly easy to use interface for Git. This blog post shows you how to start versioning your IDMS diagrams on your local machines TODAY.

In the following example, all IDMS diagrams are kept in a project called ‘Getting Started’. That project is located in the Eclipse workspace folder and is visible in both the Package Explorer and Project Explorer views.  The goal is to create a local Git repository and move the diagrams into that repository so that each committed change leaves a trace in the repository and can be used for comparisons or to be restored should the need arise.

First of all, make a backup of your Eclipse workspace folder.

Next, open the Git perspective by selecting the Window/Perspective/Open Perspective menu item and then selecting it in the list of available perspectives :

01_open_git-perspective

The leftmost view (the Git Repositories view) in the Git perspective allows you to create a new local Git repository; which is (just) a folder on your file system containing your project(s) and Git’s own folders and files needed for managing your files and versions.  Click on the Create a new local Git repository link :

02_create_new_local_repo_1

Select an empty folder on your file system AND, for performance reasons, make sure that folder is NOT a subfolder of your Eclipse workspace folder :

03_create_new_local_repo_2

After pressing the Finish button, the Git Repositories view shows you your new local repository, with the ‘NO-HEAD’ indicator next the folder name (‘IDMS’ in our example). That indicator will change to ‘master’ later on and represents the name of the branch you have checked out (I won’t discuss branching and merging here).  Apart from a folder called ‘.git’, the ‘Working Tree’ is empty because you have not yet committed any diagrams yet.

04_local_repo_created

I suggest you create a general Eclipse project in your new IDMS folder (i.e. outside the Eclipse workspace); start the Eclipse General project creation wizard via the File/New/Project… menu item.

05_create_new_general_project_1

Give your project a name (e.g. ‘diagrams’) AND make sure to uncheck the ‘Use default location’ checkbox; select the ‘IDMS’ folder on your file system :

06_create_new_general_project_2

The new project will now be visble in the Package Explorer and Project Explorer views (despite the fact that it is not stored in your workspace folder).

Copy (or move) your diagram files in either the Package Explorer or Project Explorer views.  There are several manners to accomplish this, if you want to move them, just select your diagram files and drag them to your newly created ‘diagrams’ project :

07_copy_diagram_files_1

Your Package Explorer (or Project Explorer) view should now look like this :

08_copy_diagram_files_2

You are now ready to commit your diagram files to your local Git repository.  Switch to the Git perspective by pressing the Git perspective’s toolbaar button on the top right of your screen :

09_switch_to_git_perspective

In the Git Staging view you will now see your diagram files in the ‘Unstaged Changes’ section.  Drag them to the ‘Staged Changes’ section, which represents the so-called Git index (or staging area); this is where you prepare your commits.

10_about_to_commit_1

Finally, enter a commit message in the appropriate section; press the Commit button :

11_about_to_commit_2

You have now successfully committed your first diagram in your local Git repository.  You will notice that a file called ‘.project’ is now shown in the ‘Unstaged Changes’ section; this file is needed by Eclipse and contains your project name.  Drag this file to the Git Index. We could create a separate commit for this file but Git allows you to add (and modify and remove) additional items, so press the Amend (Edit Previous Commit) button

12_about_to_commit_3

You have the chance to modify the commit message before you press the Commit button :

13_about_to_commit_4

Unless you rename your project, you will not have to go through the .project file hassle again.

Switch back to the Java perspective to reveal the Package Explorer view again :

14_switch_to_java_perspective

The nice thing about having your diagrams versioned is the fact that you can compare 2 versions, e.g. the Package Explorer (and Project Explorer) view’s context menu have a Compare With menu item, offering several options.  If you want to compare your workspace version with the last committed version, select the context menu’s HEAD Revision menu item :

15_compare_with_1

For demonstration purposes, I’ve added an email address field to the EMPSCHM schema’s EMPLOYEE record (without committing yet); when comparing the workspace version with the last committed version, a compare editor will open, showing you the differences between the 2 :

16_compare_with_2

Note that comparing 2 diagram versions will only be practical if you save your diagrams with the .schemadsl file extension; the original .schema file extension contains fairly complex XML, so comparing files with that file extension will be challenging.

The Git perspective also has a History view.  If you select the History view and then select your local repository in the Git Repositories view, you will get a list of all commits for the current branch (‘master’ in our case).  Just play around with this; you will see that you can open previous versions of your diagrams…

17_history

(note that I added some commits for demonstration purposes).

If you want to learn more about Git, you can find a lot of tutorials and books on the Internet.  This blog post was just to inform you about the possibilities it offers for you as an IDMS DBA.

Record Structures

You can change record structures with the Record Elements Import wizard.  This wizard has become more powerful in version 3.0.0 and must now be started from the Properties view (the menu item in the Diagram menu is gone forever).  There are 2 area’s in the Properties view that you can use to go ahead with modifying the structure of any record :

  • On the ‘General‘ tab :

record-general-properties

Click on the ‘Edit record element(s)‘ hyperlink to open the import wizard.

  • On the ‘Length‘ tab :

record-length-properties

You can click on either the data length or control length values to start the wizard.

Once the wizard is active, there are a few ways in which you can go ahead with changing the record structure; new in version 3.0.0 are :

  • directly specifying record elements DSL – you need to be familiar with the record element DSL.
  • connecting to a live IDMS dictionary on (one of) your mainframe(s) and use the structure of any record synonym that is defined there – note that you need to have either the SQL Option or the (free) SQL Web Connect feature for this).

import-record-elements-wizard-page-1

You will notice that, in contrast to previous versions of the schema diagram editor, the current record structure (on the wizard’s first page) is shown in record elements DSL format.  This used to be regular IDMS schema syntax, but the DSL is a litte more compact and you will thus have a better overview of the current record structure (I think).

If you choose to directly edit the record elements DSL, you will have to press the Validate button before you can proceed.  Any errors in the syntax will prevent you to advance to the final page and you will get a groovy error message that indicates your syntax is (somewhere) in error.  Pressing the Reset button undoes any changes you made.

import-record-elements-wizard-page-2a

When you go for the IDMS dictionary MO, you will be presented a list of dictionaries (that you defined in your Eclipse workspace) first, and after selecting a dictionary you will be able to search for a record synonym (no list here since that would take far too long to produce, at least if you have a lot of record synonyms in your dictionary).  I’m sorry I cannot show a screenshot of the record synonym search page (no mainframe at home, sorry).

import-record-elements-wizard-page-2b

The last wizard page gives a preview of the new record structure (in my example I changed the PICture of  the EMP-ID-0415 element and added the NEW-FIELD-0415 element).  It’s up to you to go ahead with the change  or not (by pressing either the OK or Cancel button) (but thank God you still have Eclipse’s undo/redo functionality available should you change your mind).

import-record-elements-wizard-page-3

To conclude, a word about the record elements DSL itself.  You may find that it looks a little bit like regular IDMS schema syntax and you’re right, with the difference that you MUST code every aspect for a given element on the SAME LINE.  The best advice I can give you is to look at the record elements DSL for some of your schemas and you’ll soon see that there is no magic.

Some areas of the record elements DSL need some explanation however, e.g. cases where you have base records, for which you define record synonyms (with suffixes or prefixes, but I’m not sure how well the schema diagram editor handles the latter), that you use in your schemas (this is how we’ve worked for the last 30 years or so).  You’ll see that the base element name will appear in parentheses following the element (synonym) name, like so :

02 IDMS-RELEASE-7301 (IDMS-RELEASE) picture 9

Other cases involve indexes on OCCURS clauses :

02 ELEMENT-1 picture X(8)
   occurs 5 indexedBy ELEMENT-WITHOUT-BASE-NAME, 
                      ELEMENT-WITH-BASE-NAME-1
                         (INDEX-ELEMENT-BASE-NAME-1), 
                      ELEMENT-WITH-BASE-NAME-2
                         (INDEX-ELEMENT-BASE-NAME-2)

Remember to code everything that belongs to 1 element on the same line, so just assume that all of the above is on the same line.

I believe the diagram editor is now (and finally) in a stage that it can be used to design IDMS databases from scratch and edit existing ones in every facet.

Version 3.0.0 is out

It’s been a while since the previous version (2.1.0) was published, but I’ve been working quite hard on this new version for the last 5 months or so.

Why a new major version ?  Because the DSL that was introduced in version 2.1.0 now has its own file extension :  .schemadsl.  You can now save diagrams in this new file format and you’ll see that it is a LOT more human readable than the .schema file extension that was (and still is) the standard.  A drawback of the new file format is that it takes longer to load a diagram; not such a big deal for small diagrams, but I’m sure you’ll notice the difference for IDMSNTWK. The big advantage however is that it now makes sense to compare a .schemadsl diagram with a previous version (which you’ve put aside or kept in a version control system like Subversion or GIT) AND you can tweak the file yourself. You can set the default file extension in the preferences and can easily switch between the 2 using the File menu’s ‘Save As…’ menu item (just change the file extension).

Another big improvement in the new version is the ability to manage a record type’s elements :  besides copying the structure from another record in 1 of your diagrams, you can now enter the record elements DSL yourself or connect to a CA IDMS dictionary and use a record synonym’s elements.

An overview of all what’s new and noteworthy in (GIT) commit comment style :

  1. Record element management changes :
    • Add ‘Edit record element(s)’ bottom hyperlink to general record properties
    • Invoke import record elements wizard from record length property section
    • Replace schema syntax by rec. el. DSL on import rec. elements wizard’s 1st page
    • Add the preview page to the import record elements wizard
    • Develop the ‘Import from DSL’ record elements import tool
    • Develop the import record elements from a mainframe dictionary tool
    • Set help available to false for the import record elements wizard
    • Change undo/redo ‘Import Record Elements’ to ‘Edit Record Elements’
  2. DSL changes :
    • Suppress area body in schema DSL when empty (e.g. area DDLDCTRC in IDMSNTWK)
    • Fine-tune the area and record procedure call related DSL
    • Restyle the record elements DSL syntax
  3. Groovy :
    • Upgrade Groovy to v2.4.7
  4. Miscellaneous changes :
    • Implement the ‘edit schema comments’ feature
    • Develop the dialog to maintain an area or record’s procedure call specs
    • Provide read-only feature for the diagram editor
    • Diagrams can be opened from the SVN Repositories view (read-only)
    • Add support for the .schemadsl file extension
    • Add diagram preferences page and default file extension preference
    • Adapt New Schema Wizard to take default file extension into account
    • Adapt Import/Update Schema Wizard to take default file extension into account
    • Closing all .schemadsl files on workbench shutdown is configurable
    • Provide model diagram for the current Ecore Diagram Editor
  5. Bug fixes :
    • Rulers are ignored in the DSL and are lost when saving to .schemadsl
    • Prevent NullPointerException when saving with Eclipse Neon

I’ll try to elaborate on some of these items in future posts…

Reworked record DSL

The next version of the diagram editor is in development.  Actually, development work for that next version was started right after publishing the current version, some 4 months ago already, but was paused after about a month.  I’ve been working on a (private) project that I’m determined to deliver some day (for myself) in the future :  my own version of Microsoft Money, a product that is no longer in support for a number of years and that I’m still using for my personal finances.  For those interested, it is an Eclipse e4 RCP application that I’m writing entirely in Groovy and that is backed by a Neo4j graph database.

A first feature of the new version will be the ability to view a diagram directly from the SVN Repository Exploring views of both Subclipse and Subversive (both are Subversion clients, needed for version control; you usually install only one of these when working with Subversion).  Subclipse is the one we use at the office and some of our developers, while switching to modern tools like Eclipse and the Compuware/Topaz Workbench for their daily development tasks, have brought this issue to light.  Before this feature, where the diagram editor actually becomes a diagram viewer (the introduction of read-only mode), you needed to checkout the whole project containing the diagram of interest, thus soon you will be able to view a diagram directly from your Subversion repository.

Knipsel

The above image shows the IDMS demo database schema opened via the Subversive plug-in.

The ‘schema’ file extension is currently the only file format supported for storing your diagrams.  That format is the easiest way to save Ecore models (like your diagrams) from a programmatic point of view, but the resulting file (XML/XMI) is very hard to read and tweak by a human being. For that reason, the ‘schemadsl’ file extension will be introduced.  You will be able to choose the file format when creating/saving/importing diagrams in the next version.  The contents of the ‘schemadsl’ file format is the DSL that was introduced in version 2.1.0.  You will see that that format is far more understandable and tweakable than the ‘schema’ file format. The syntax for defining a record’s elements will probably change and become more terse, as in the following screen shot :

New record DSL

What I hope to achieve with this new DSL syntax is – apart from better readability, better performance, because loading the IDMSNTWK schema from its DSL syntax takes a lot longer than loading the same diagram from its ‘schema’ file…