Inputs¶

Overview¶

Every job in Supplier Data Manager (SDM) starts with an input file. By default, when using a file type that supports multiple sheets (such as Excel), only the first sheet is processed. For CSV files, SDM auto-detects the column separator and reads the file from the first header row.

Custom Input Formats let you override this default behavior. They are predefined templates that specify exactly how a file should be matched and parsed — which sheets or columns to include, how to join data across sheets, and what identifiers to use. They support more complex data imports that cannot be handled with a plain file upload.

For a one-off complex Excel file, users can configure the file directly with the File Format wizard. Save the Wizard configuration as a Custom Input Format when the same structure will be imported again.

SDM supports three types of Custom Input Format:

Excel — for multi-sheet .xlsx/.xls files. Defined in the admin panel and applied automatically when an uploaded file matches.
CSV — for .csv files that require column remapping, encoding configuration, or uniqueness constraints.
XML — for structured XML files. See XML files and Custom Input Formats for full configuration details.

Custom Input Formats are managed in the SDM admin panel at Home > Files > Input formats.

The screenshot above shows the Custom Input Formats list in the Supplier Data Manager admin panel, displaying each configured format with its name, organization, and type.

Input upload process¶

The following diagram illustrates how an uploaded file is matched against Custom Input Formats and parsed into the dataframe used by the job.

The diagram above shows the Supplier Data Manager input upload process: the system checks each configured Custom Input Format in turn, matches the file against the format's criteria, and transforms the data into a dataframe for downstream job steps.

When a file is uploaded, SDM tries each Custom Input Format defined for the organization. The first matching format is applied. If no Custom Input Format matches, SDM falls back to the default single-sheet behavior.

Excel Custom Input Formats¶

Excel Custom Input Formats apply to .xlsx and .xls files. They allow you to define which sheets to load, which columns to extract from each sheet, and how to join sheets together into a single dataframe.

Warning

The configuration described in this section is a readable summary of the API docs. The API docs are always up to date and are the authoritative source. Verify the information here against them when in doubt.

The screenshot above shows the Excel Custom Input Format configuration form in the Supplier Data Manager admin panel, with fields for name, organization, and the JSON params object.

Structure¶

The configuration object for an Excel Custom Input Format has the following top-level structure:

{
  "sheets": [ ... ],
  "unique_columns": [ ... ]
}

Sheet options¶

The sheets property is a required array of objects. Each object defines one sheet in the Excel file and supports the following properties:

name (string, required): The name of the sheet in the Excel file. Leading and trailing whitespace in the sheet name is ignored during matching.
columns (array, required): Specifies which columns to extract from the sheet. Each entry in the array can be one of:
Standard column — { "name": "ColumnName" }: extracts the named column.
Name-value pair column — { "name_from": "AttrName", "value_from": "AttrValue" }: pivots rows into columns using the values in name_from as column names and value_from as column values.
Wildcard — null: includes all remaining columns not otherwise specified.

Warning

You cannot use name and name_from/value_from in the same column object.

skip_rows (integer, optional): Number of rows to skip at the beginning of the sheet before reading the header. Useful for files with preamble rows above the column headers.
required (boolean, optional, default: true): When true, the sheet must be present in the file for the format to match. When false, the sheet is optional.
load (boolean, optional, default: true): When true, the sheet's data is included in the resulting dataframe. Set to false to use a sheet only for format matching without loading its data.
drop_duplicated_columns (boolean, optional, default: false): When true, removes columns that appear in multiple sheets (keeping the first occurrence). Useful when the same identifier column is repeated across sheets.
read_rich_text_formatting (boolean, optional, default: false): When true, reads rich text formatting (such as bold or italic) from cells in addition to their values.
join (object, optional): Specifies how this sheet should be joined with the data loaded so far. Properties:
on (array of strings): The column(s) to join on.
how (string): The join type — "left", "right", "inner", or "outer".

Info

Joins use pandas DataFrame.join under the hood. Refer to the pandas documentation for details on join behavior.

Unique_columns option¶

The unique_columns property is an optional array of strings listing the column names that serve as unique identifiers across all sheets. SDM enforces uniqueness on these columns in the resulting dataframe.

Data transformation¶

Once a file matches an Excel Custom Input Format in Supplier Data Manager:

SDM reads each sheet listed in sheets, in order, skipping any optional sheets that are absent.
For each sheet, it extracts the columns specified in columns. If null is present in columns, all unnamed columns are also included.
Sheets with a join definition are merged into the running dataframe using the specified join type and key columns.
Duplicate columns are removed if drop_duplicated_columns is true.
Uniqueness is enforced on columns listed in unique_columns.

The resulting dataframe is passed to the first step of the SDM job.

Excel Custom Input Format example¶

{
  "sheets": [
    {
      "name": "Products",
      "columns": [
        { "name": "ProductID" },
        { "name": "ProductName" },
        { "name": "Category" }
      ],
      "skip_rows": 2,
      "required": true
    },
    {
      "name": "Attributes",
      "columns": [
        { "name": "ProductID" },
        { "name_from": "AttributeName_3", "value_from": "AttributeValue_3" }
      ],
      "drop_duplicated_columns": true,
      "join": {
        "on": ["ProductID"],
        "how": "left"
      }
    }
  ],
  "unique_columns": ["ProductID"]
}

CSV Custom Input Formats¶

CSV Custom Input Formats apply to .csv files. They let you define the file encoding, column separator, column remapping, and uniqueness constraints. When an uploaded CSV file matches the format's column requirements, the format is applied automatically.

CSV configuration options¶

The configuration object for a CSV Custom Input Format uses the following properties:

encoding (string, optional, default: "utf-8"): The character encoding of the CSV file. Examples: "utf-8", "latin-1", "cp1252".
separator (string, optional): The column separator character. If not provided, SDM auto-detects the separator from the file content.
skip_rows (integer, optional): Number of rows to skip before the header row.
columns (array, optional): Defines column extraction and remapping, using the same syntax as Excel sheet columns:
{ "name": "ColumnName" } — include a column by its original name.
{ "name": "NewName", "value_from": "OriginalName" } — rename a column.
{ "name_from": "AttrName", "value_from": "AttrValue" } — pivot name-value pairs into columns.
null — include all remaining columns.
unique_columns (array of strings, optional): Column names to enforce uniqueness on in the resulting dataframe.

CSV Custom Input Format example¶

{
  "encoding": "latin-1",
  "separator": ";",
  "columns": [
    { "name": "REFCIALE" },
    { "name": "ProductName", "value_from": "LIBELLE240" },
    null
  ],
  "unique_columns": ["REFCIALE"]
}

This format reads a semicolon-separated file encoded in latin-1, renames the LIBELLE240 column to ProductName, includes all remaining columns, and enforces uniqueness on REFCIALE.

XML Custom Input Formats¶

XML Custom Input Formats apply to .xml files. They require a separate configuration using root paths, column extraction, and namespace handling options.

Warning

Adding a Custom Input Format for your expected XML files is mandatory — SDM cannot process XML files without one.

For full configuration details, see XML files and Custom Input Formats.

Use case¶

Consider a supplier who delivers an Excel file with two sheets: one for commercial data and one for logistics data. The following Excel Custom Input Format in Supplier Data Manager handles this structure:

{
  "sheets": [
    {
      "name": "01_COMMERCE",
      "columns": [
        { "name": "REFCIALE" },
        { "name": "LIBELLE240" },
        { "name": "LIBELLE80" },
        { "name": "GTIN13" },
        null
      ],
      "required": true
    },
    {
      "name": "02_LOGISTIQUE",
      "columns": [
        { "name": "MARQUE" },
        { "name": "REFCIALE" },
        null
      ],
      "required": true,
      "join": {
        "on": ["REFCIALE"],
        "how": "left"
      }
    }
  ],
  "unique_columns": ["REFCIALE"]
}

This format:

Matches files that contain sheets named 01_COMMERCE and 02_LOGISTIQUE.
Verifies that the specified columns exist in each sheet.
Includes all columns from both sheets in the final dataset (due to the null wildcard in columns).
Joins the 02_LOGISTIQUE sheet onto 01_COMMERCE using a left join on the REFCIALE column.

Implications for downstream job steps¶

The resulting dataframe contains only the data specified by the Custom Input Format.
Removing null from the columns array restricts the dataframe to only the explicitly named columns.
All job steps — extraction, mapping, normalization, and others — receive this dataframe as their input.