We can encounter several failures while importing content/data in Asta7 for several reasons. So far our approach for handling failures has been to throw an exception and abort the import. But this approach is not user-friendly as most of the failures are recoverable.
...
| Info |
|---|
This tool will be used only for the Single Archive and Excel import from Asta7 |
Failures and Possible Solutions
Normal Entities
...
Failure
...
Solutions
...
Challenges
...
AMID conflict
...
Regenarate
...
Replace all
objectReference
...
PK conflict (Single PK only for now)
...
Regenaret if UUID
Input from the user
...
Replace all
FKHow to handle composite PK?
...
Unique conflict
...
Regenarate if UUID
Input from the user
Set
nullif nullable
...
FVP Unique conflict (Root entities only for now)
...
Regenarate if FVP
Input from the user
Set
nullif nullable
...
FK conflict (Not yet supported)
...
Set
nullif nullable
...
NULL conflict
...
Regenarate if UUID
Regenarate if FVP and root
Input from the user
...
Wrong type
...
Regenarate if UUID
Regenarate if FVP and root
Input from the user
Set
nullif nullable
...
Wrong length
...
Input from the user
Trim extra
Set
nullif nullable
System Entities
For this section, we need to consider the system root entities like AKTOR, TAG, etc. and ignore AMID/PK conflicts for all sub-entities. We should add some options for handling AMID/PK/Unique/FVP Unique conflicts for the system root entities.
...
Failure
...
Solutions
...
Challenges
...
AMID/PK/Unique/FVP Unique conflict
...
Ignore
Replace with the conflicted row
...
If ignored then need to ignore the whole subtree for this row as well
...
If replaced need to replace all objectReference and FK as well
...
| Table of Contents | ||
|---|---|---|
|
Conflicts
Type | Description | Solutions | Example |
|---|---|---|---|
UNIQUE |
|
| There is an object of entity A with AMID There is an entity A which has a field |
FK |
|
| Entity A has a sub-entity Entity B. Entity B then will have a field like |
REQUIRED |
|
| Entity A has a primary key field |
TYPE |
|
| Entity A has a field |
LENGTH |
|
| Entity A has a |
Root System Entity UNIQUE |
|
| The system already has an AKTOR with |
Solutions
Type | Description | Challenges |
|---|---|---|
GENERATE |
|
|
SET_BY_USER |
|
|
SET_NULL |
| |
SET_NULL_OR_REGENERATE |
| |
TRIM_EXTRA |
| |
IGNORE |
|
|
REPLACE_WITH_EXISTING |
|
|
FAIL |
|
Implementation
We can divide the implementation into four key steps.
...
As before we will try to import the data without any changes. In case of an exception, we will analyze the exception to check if it is a resolvable exception (AMID/PK conflict, NULL conflict, etc. ). If resolvable then the import job dialog will show a button to start the resolving process.
...
Clicking the Resolve button will start the 2nd step.
...
Conflict Resolution Options
After clicking the Resolve button a new tab will be opened. The tab will start a background job In the tab, there will be options to choose how do you want to proceed. There will be options present per conflict type. Either you can gather the conflicts beforehand, or gather and solve them while importing, and the third option is to mix both, meaning gather some types beforehand but resolve the rest while importing.
...
If all the types are set to resolve at import time then the Resolve and Import button will be shown. In this case, the 3rd step is not needed anymore. Otherwise, you can proceed to the 3rd step by clicking the Gather Conflicts button.
Gather and Show Conflicts with Possible Solutions
After clicking the Gather Conflicts button a background job will be started, which will go through the import file and gather all the resolvable conflicts for the chosen types. After gathering all the conflict the 3rd step will begin.
Show Conflicts and Possible Solutions
Once the conflict-gathering job is done the tab will show all the conflicts in a table.
...
The table contains the following columns
...
Column
...
Description
...
Entity Name
...
they will be shown in a table with possible solutions to choose from. The conflicts will be grouped by the entity name and the conflict type. It is also possible to pick a solution for the whole group.
| Info |
|---|
The current limit for gathering conflicts is 10,000. |
...
Resolve and Import
Clicking the Resolve and Import button will begin the 4th and final step. In this step, the import will run again. All the conflicts gathered beforehand and conflicts gathered while running will be resolved based on the chosen solution.
Things To Do
Support Excel import
Support Composite PK and FK
Generate other types than UUID
Optimize using temporary table and/or files rather than the memory
Add more types of solutions