# The IngestConfig Class

The IngestConfig object is the declarative heart of your importer. It allows you to define the entire ETL (Extract, Transform, Load) process in a fluent, readable way.

All configuration happens inside the getConfig() method of your importer class.

Required. Initializes the configuration for a specific Eloquent model. This must be the static entry point.

IngestConfig::for(\App\Models\Product::class)

Required. Defines where the data comes from.

• $type: An enum instance of LaravelIngest\Enums\SourceType.
• $options: An associative array of options required by the specific source handler (e.g., path, disk, url).

->fromSource(SourceType::FTP, ['disk' => 'ftp-disk', 'path' => 'import.csv'])

Defines the "Unique ID" column in your source file (not the database column name). This is used to check if a record already exists.

// The CSV has a column "EAN-Code" which is unique
->keyedBy('EAN-Code')

Defines behavior when a record with the keyedBy value is found in the database.

• DuplicateStrategy::SKIP: (Default) Do nothing. Keep the old record.
• DuplicateStrategy::UPDATE: Overwrite the database record with new data.
• DuplicateStrategy::FAIL: Stop processing this row and mark it as failed.
• DuplicateStrategy::UPDATE_IF_NEWER: Only update if the source data is newer (requires compareTimestamp()).

->onDuplicate(DuplicateStrategy::UPDATE)

Used with DuplicateStrategy::UPDATE_IF_NEWER. Compares a timestamp from the source data with a database column to determine if the record should be updated.

->onDuplicate(DuplicateStrategy::UPDATE_IF_NEWER)
->compareTimestamp('last_modified', 'updated_at')

A 1:1 copy from source to database. Supports column aliases for files with varying headers.

// Simple mapping
->map('First Name', 'first_name')

// With aliases - first match wins
->map(['email', 'E-Mail', 'user_email'], 'email')
->map(['name', 'full_name', 'Name'], 'name')

Transforms the value before saving. Also supports column aliases.

• Callback Signature: fn($value, array $row)
• $value: The value of the specific column.
• $row: The entire raw row array (useful for combining columns).

// Combine First and Last name
->mapAndTransform('Last Name', 'full_name', function($value, $row) {
    return $row['First Name'] . ' ' . $value;
})

// Format currency
->mapAndTransform('Price', 'price_in_cents', fn($val) => (int)($val * 100))

// With aliases
->mapAndTransform(['status', 'Status', 'STATE'], 'is_active', fn($val) => $val === 'active')

Automatically resolves BelongsTo relationships.

1. Takes the value from $sourceColumn.
2. Searches $relatedModel where $relatedKey matches that value.
3. If found, assigns the ID to the foreign key of $relationName.
4. If createIfMissing is true and no match is found, creates the related record automatically.

// Source: "Category: Smartphones"
// Database lookup: Category::where('name', 'Smartphones')->first()
// Result: $product->category_id = $foundCategory->id
->relate('Category', 'category', \App\Models\Category::class, 'name')

// Auto-create missing categories
->relate('Category', 'category', \App\Models\Category::class, 'name', createIfMissing: true)

Synchronizes Many-to-Many relationships from a delimited list in your source data. Perfect for tags, categories, or any pivot table relationship.

Parameters:

• $sourceField: Column name in your source file containing the delimited values
• $relationName: Name of the BelongsToMany relationship in your model
• $relatedModel: The related Eloquent model class
• $relatedKey: Attribute to search for in the related model (default: 'id')
• $separator: Character used to split values (default: ',')

// Example 1: Tags from CSV column "Tags" containing "PHP, Laravel, Backend"
->relateMany(
    sourceField: 'tag_names',       // Spalte im CSV (z.B. "Laravel,PHP,API")
    relation: 'tags',                // Name der Beziehung im Model
    relatedModel: Tag::class,        // Klasse des verwandten Models
    relatedKey: 'name',              // Attribut zum Suchen/Aufösen
    separator: ','                   // Trennzeichen (Default: ",")
)

// Example 2: Categories with semicolon separator
->relateMany(
    sourceField: 'categories',
    relation: 'categories',
    relatedModel: Category::class,
    relatedKey: 'slug',
    separator: ';'
)

// Example 3: Multiple role assignments
->relateMany(
    sourceField: 'user_roles',
    relation: 'roles',
    relatedModel: Role::class,
    relatedKey: 'name',
    separator: '|'
)

Behavior:

• Duplicates in the source list are automatically handled
• Non-existing related records will cause the row to fail (unless you handle them in beforeRow())
• The entire pivot table for the relationship is synced (existing relationships not in the list will be removed)

---

## Validation

### `validate(array $rules)`
Applies Laravel validation rules to the incoming data *before* it is transformed or saved. Keys must match the **source file columns**.

```php
->validate([
    'EAN-Code' => 'required|numeric|digits:13',
    'Price' => 'required|numeric|min:0',
])

Merges validation rules defined in the target model's static getRules() method. Useful for DRY (Don't Repeat Yourself). Rules from validate() take precedence over model rules.

// In Product.php
public static function getRules(): array
{
    return [
        'sku' => 'required|string',
        'name' => 'required|min:3',
    ];
}

// In Config
->validateWithModelRules()
->validate(['price' => 'required|numeric']) // Additional rules

Executed before validation. Allows you to modify the raw data array by reference. Perfect for cleaning up messy data globally.

->beforeRow(function(array &$data) {
    // Remove invisible characters from all keys
    $data = array_combine(
        array_map('trim', array_keys($data)), 
        $data
    );
})

Executed after the model has been successfully saved.

• $model: The saved Eloquent model.
• $row: The original raw data.

->afterRow(function(Product $product, array $row) {
    // Sync tags or trigger side effects
    $product->search_index_updated_at = now();
    $product->saveQuietly();
})

Determines how many rows are processed per background job. Default: 100.

• Increase for simple inserts to reduce queue overhead.
• Decrease for memory-heavy operations (e.g., image processing in afterRow).

Wraps each chunk in a Database Transaction. If one row in the chunk fails, all rows in that chunk are rolled back.

• Default: Disabled (Rows are committed individually).

Overrides the default filesystem disk (from config/ingest.php) for this specific importer.

->setDisk('s3_private_bucket')

Enables strict header validation. When enabled, the import will fail immediately if any mapped source column is missing from the file headers. By default, only the keyedBy column is validated.

->strictHeaders()
->map('email', 'email')      // Must exist in source file
->map('name', 'name')        // Must exist in source file

Allows you to dynamically determine which Eloquent model to use based on the row data. This is useful when importing heterogeneous data into different tables.

• Callback Signature: fn(array $rowData): string
• Returns: A fully qualified model class name.

use App\Models\{User, AdminUser, Customer};

IngestConfig::for(User::class)
    ->resolveModelUsing(function(array $row) {
        return match($row['user_type'] ?? 'user') {
            'admin' => AdminUser::class,
            'customer' => Customer::class,
            default => User::class,
        };
    })
    ->map('email', 'email')
    ->map('name', 'name');

Note: The base model class passed to IngestConfig::for() is used as a fallback if no resolver is set.

Fine-grained control over database transaction behavior.

• TransactionMode::NONE: No transactions (default). Each row is committed individually.
• TransactionMode::CHUNK: Wraps each chunk in a transaction. Same as calling atomic().
• TransactionMode::ROW: Wraps each individual row in its own transaction.

use LaravelIngest\Enums\TransactionMode;

->transactionMode(TransactionMode::ROW)

Aktuell unterstützt keyedBy() nur einfache Schlüssel. Für zusammengesetzte Schlüssel (z.B. ['store_id', 'sku']) können Sie Workarounds verwenden:

Erstellen Sie eine kombinierte Spalte in Ihrer Quelldatei:

// CSV enthält: store_id, sku, product_name
// Erstellen Sie eine neue Spalte: unique_key = "store_id|sku"
->keyedBy('unique_key')

Beispiel CSV-Transformation:

store_id,sku,product_name,unique_key
1,PROD001,Product A,"1|PROD001"
2,PROD001,Product B,"2|PROD001"

Verwenden Sie die beforeRow() Methode, um einen kombinierten Schlüssel zur Laufzeit zu erstellen:

->beforeRow(function(array &$row) {
    // Kombiniere store_id und sku zu einem einzigartigen Schlüssel
    $row['composite_key'] = ($row['store_id'] ?? '') . '|' . ($row['sku'] ?? '');
})
->keyedBy('composite_key')

Für komplexe Szenarien können Sie die Daten vor dem Import in einem separaten Prozess vorbereiten:

// In einem Service oder Controller:
public function prepareImportData(string $inputPath, string $outputPath): void
{
    $csv = array_map('str_getcsv', file($inputPath));
    $headers = array_shift($csv);
    
    $prepared = [];
    foreach ($csv as $row) {
        $row = array_combine($headers, $row);
        $row['store_sku_key'] = $row['store_id'] . '_' . $row['sku'];
        $prepared[] = $row;
    }
    
    // Schreibe bereinigte CSV
    $fp = fopen($outputPath, 'w');
    fputcsv($fp, array_keys($prepared[0]));
    foreach ($prepared as $row) {
        fputcsv($fp, $row);
    }
    fclose($fp);
}

Hinweis: Echte Composite-Key-Unterstützung ist für Version 0.5 geplant und wird diese Workarounds überflüssig machen.