How to Use the Data Center to Import Data into Soley Studio?

The Data Center feature allows for a simplified import of data from files (Excel, .csv, and .slg) or from various databases without writing a line of code. Based on the structure of the data table, you can interactively select the parts of the table which should be imported and map the columns (or rows) of the table to the corresponding attributes defined in the metamodel. This makes the Data Center a very flexible tool for importing data from multiple sources into Soley Studio. This article gives you a quick step by step guide on how to import data using the Data Center feature.

The basic process of the data center is as follows:

  1. Define a data source, i.e. a file or database that you want to import from
  2. Define an import step, i.e. a mapping of the data to a node class and it’s attributes
  3. Import the data directly or define an import sequence to run several import steps in one go
Basic process of using the data center to import data and create a graph out of it
Basic process of using the Data Center to import data into a graph

The Data Center will soon replace the Data Import Dialog. For the transition phase, we keep both options active.

Adding a new data source

A data source defines a file or database that you want to import from. To add a new data source to your project, go to the top menu tool bar and click on the icon “Add New Data Source…”.

Icons for Data Center plus description
Icons for importing data using the Data Center

Now, the “Select data connection” dialog pops up, in which you can select the data source. This can be an .xlsx, .csv, .slg file, a database, a PLM/ERP/CRM system or an online service. Some of the interfaces are currently under development, some are extra add-ons. Select an entry from the list and click on Connect.

Select data connection dialog
Select data connection dialog

The connections of Soley Software include i.e. SQL (MySQL, PostgreSQL, SQLite, Microsoft SQL), Oracle, IMB Databases, Sybase and Hadoop. Furthermore, you can connect Soley Studio directly to a variety of PLM systems (i.e. Windchill ptc, Teamcenter Siemens As and many more), ERP (i.e. SAP Database, Microsoft Dynamics NAV/GP/AX, Odoo) or even CRM systems (i.e. Microsoft Dynamics CRM, Salesforce Objects, Zendesk, Odoo) and online services (i.e. Google Analytics). Click here for more detailed information about available data connections. As soon as you have selected the data source, a new *.slds file is added automatically to the subfolder Data Sources, which you can find in the Solution Explorer under Data Center.

Adding a new import step

Select the icon “Add New Import Step…” in the top menu tool bar. This will open up a dialog window showing all data sources you added. Select the source you want to import data from and click OK.

In the appearing window, you can define the details of the import, i.e. pick a worksheet, select the node class to be instantiated and map the attributes to the columns. Let us have a look at an example.

We want to import data from an Excel file and already added the data source as described in step 1. After clicking “Add New Import Step…” the following dialog window appears.

Define the import in the import step dialog

  1. Select the Worksheet (input field “Worksheet”). This only applies if you import from Excel.
  2. Define the class of the nodes you want to import (input field “Node class”). If you want to create a new node class you can use In-Place Metamodeling. Note that you can only define one node class per import step. You can combine multiple import steps by adding them to one import sequence (see below).
  3. Map the attributes of the selected node class to the columns of the data table. Use the dropdown menus to see all available attributes and make a selection. You can map one column to multiple attributes. If you want to create a new attribute for an existing node class or a newly created node class you can use In-Place Metamodeling.
  4. Mark the import area of the table (blue area in the picture). Make sure you leave out the header row and select only rows which contain information about the node instances you want to create (or update). If you want to import all data, select the first row containing data and then tick the “Always read till the last row” option (tick box under the data table).
  5. In the Settings tab, in the Import section, you can chose between “Create” (this import will create new nodes), “Update” (this import will update existing nodes), or “UpdateOrCreate” (this will either update existing nodes or create the ones which don’t exist yet).
    This update will be done according to a certain Id attribute, to identify existing nodes. This Id attribute should be of type int, long, short, byte or string.

    Settings of a new Import Step Import Step dialog window
    Settings of a new Import Step

    This is very useful if you want to map attributes from different Data Sources to the same nodes. You can simply import the nodes once, add the attributes in different import steps, and use all steps in an Import Sequence at the end.

  6. Under Data, you can define separators if your cells contain arrays or maps. E.g. if your cells contain multiple strings separated by a comma, you can directly import them as an array into Soley Studio, after defining the comma as separator.
  7. Click OK to save the Import Step. The Data Center dialog will ask you if you want to import the data right away. If you have a graph open you will be asked whether to import into the open graph or into a new graph. You can also decide not to import right away, e.g. if you want to create an Import Sequence with several Import Steps or if you want to run the Import Step from your project code.

In any case, a new *.slis file is created automatically and attached to the selected data source in the Solution Explorer. In our example, it is attached to the file “Soley_PPA_ImportData01.slds”. To edit the Import Step you may open the *.slis file.

Using the load-mapping option: 

The load-mapping button is useful when you have two or more similar files that you want to import. By similar is meant order of columns and attributes. All you have to do is create the metamodel once for the first file (node class, attributes and area to map) and then open your second file and load the mapping of the first one, which means selecting the import-step of the first file.

Optional: Create and run an import sequence

Right-click on the folder “Data Center” in the Solution Explorer and select “Add New Import Sequence”. In the appearing import sequence dialog, you see all import steps (.slis files) you added to the folder “Data Sources”. You can freely combine the import steps you previously defined. In the following example, we want to embed two import steps in the same import sequence. Thus, when carrying out the import sequence, all import steps contained are carried out automatically one after another. It is important that if you are updating an existing graph file that you put this after you have already created said graph file in the order of importing steps.

By clicking Ok, a new subfolder “Import Sequences” is added automatically to the folder “Data Center”. This folder contains all import sequences (*.slim files) you defined.

Run the import sequence

Right-click on the *.slim file in the folder “Import Sequences” and select “Run Import Sequence”. By default, this will open up a new graph file and carries out all the import steps contained in the sequence. If you want to import your data into an existing graph, open the graph file (.slg) in advance.

If you run an import sequence while a graph file is open, you can chose whether you want to import the node instances to the existing graph or not (and therefore create a new graph).

 

X) Troubleshooting

To run an import via Data Center it is necessary that the folder “Data Center” and its subfolder “Data Sources” exist in the Solution Explorer on the left side of the Soley Studio window. In later versions of Soley Studio, the folders are pre-set. If your folder structure in the Solution Explorer does not look like this…

Setting up the folder structure for using the Data Center in Soley Studio.
Required folder structure for using the Data Center in Soley Studio

e.g. if you are working on an older project or deleted one of the folders, please create them by right-clicking on the project name in the Solution Explorer –> Add –> New Folder. Name it “Data Center”. Then, create a subfolder of “Data Center” by right-clicking on “Data Center” –>Add –> New Folder and name it “Data Sources”. The folder “Import Sequences” will be created automatically later on.

Outlook

The Data Center is a new feature we are currently working on. Upcoming improvements include usability, UI and ways to call import sequences within sequences and workflows. We will update this article as soon as the improvements are available.

Was this article helpful?

Related Articles