Import

The import module of the anyPIM enables bulk ingestion of product data via standardized Excel files. It is designed for product managers and data stewards who want to efficiently load large volumes of data into the system -- without requiring technical API knowledge.

Concept

The import follows the principle of "validation before execution": data is never written directly to the database without being fully validated first. This ensures that faulty records do not cause inconsistent states.

Three-Phase Import Process

The import goes through three clearly separated phases:

Phase 1: Upload

The user uploads an Excel file via the import interface. The system automatically detects the contained worksheets (tabs) and maps them to the internal data types. The file is stored server-side and is ready for validation.

Phase 2: Validation

During the validation phase, the system checks each record for:

• Schema conformity -- Required fields, data types, valid enum values
• Reference resolution -- Technical names are resolved to UUIDs
• Dependencies -- Referenced entities must exist (in the file or in the system)
• Duplicate detection -- Already existing records are marked for upsert
• Fuzzy matching -- Typos in references are detected and correction suggestions are displayed

The result is a detailed validation report with errors, warnings, and a preview of the planned changes (diff view: create, update, skip).

Phase 3: Execution

Only after successful validation and explicit confirmation by the user are the data written to the database. The execution is transaction-safe -- either all records of a tab are successfully processed or the entire tab is rolled back. After completion, the user receives a result report.

Excel Template: 14 Tabs

The import file consists of 14 worksheets that are processed in a defined order and dependency structure:

No.	Tab	Description
01	Languages	Available content languages
02	Attribute Groups	Logical grouping of attributes
03	Unit Groups	Groups of physical units
04	Units	Individual units of measurement with conversion factors
05	Attributes	Attribute definitions (19 columns)
06	Hierarchies	Master and output hierarchies with up to 6 levels
07	Value Lists	Predefined selection values for attributes
08	Products	Product master data (SKU, name, type, status)
09	Product Values	Attribute values per product, language, and repetition
10	Variants	Product variants with assignment to parent product
11	Variant Values	Attribute values of variants
12	Media	Media assignments to products
13	Prices	Price information with currency and validity
14	Relations	Relationships between products

Note

Not all tabs need to be filled. You can selectively use only the tabs that are relevant for your import. Dependencies are resolved both within the file and against the existing data inventory.

Upsert Logic

The import operates on the upsert principle (Update or Insert):

• New record: Is created if no record with the same identifier (e.g., technical name, SKU) exists.
• Existing record: Is updated if a record with the same identifier already exists. Only the fields contained in the import file are overwritten.

This behavior enables both initial data migrations and incremental updates using the same import file.

Smart Matching

The system uses intelligent matching to automatically detect typos in references:

• Levenshtein distance with a threshold of 85%
• Case-insensitive comparison
• Whitespace trimming before comparison

If a similar but not exactly matching value is found, the system displays a correction suggestion in the validation preview.

BMEcat Import

In addition to the Excel import, the anyPIM supports the import of product data in BMEcat format (versions 1.2 and 2005). BMEcat is a standardized XML format for the electronic exchange of product catalogs, widely used in B2B commerce.

Supported BMEcat Versions

Version	Standard	Description
BMEcat 1.2	Classic	Widely used, especially in German-speaking countries
BMEcat 2005	Current	Extended structure with additional elements

Import Modes

The BMEcat import supports two modes:

Mode	Description
Update	Existing data is updated, new records are created (upsert)
Delete & Insert	All existing data of the affected entities is deleted and re-imported

Imported Data

The following BMEcat elements are processed:

• Product types and product master data (PRODUCT)
• Attributes from FEATURE/FNAME elements
• Prices from PRODUCT_PRICE_DETAILS
• Hierarchies from CATALOG_GROUP_SYSTEM
• Hierarchy attribute assignments
• Product-hierarchy assignments (PRODUCT_TO_CATALOGGROUP_MAP)
• Product relationships (PRODUCT_REFERENCE)
• Media references (MIME_INFO)

Validation

The BMEcat import offers a validation mode that checks the XML file without importing data:

POST /api/v1/bmecat-import/validate

The validation detects missing required fields, invalid references, and structural errors in the XML.

Technical Details

• Streaming processing -- The XML file is parsed using XMLReader to efficiently process even large files with minimal memory usage.
• Transaction safety -- The import is transaction-safe.

Further Documentation

• Excel Format -- Detailed column documentation for all 14 tabs
• Validation -- Validation rules, error messages, and preview view