# Custom Source Handlers

Sometimes CSVs aren't enough. You might need to import data from a JSON API, an XML feed, or a Google Sheet. You can add support for any data source by creating a custom Source Handler.

# The Concept

A Source Handler is responsible for one thing: Converting a raw source into a Generator of arrays.

Laravel Ingest doesn't care where the data comes from, as long as you yield it row by row. This ensures memory efficiency even for massive datasets.

# The SourceHandler Interface

Every handler must implement LaravelIngest\Contracts\SourceHandler:

interface SourceHandler
{
    /**
     * Read data from the source and yield rows as arrays.
     * @param mixed|null $payload Data from the trigger (e.g., UploadedFile, path, or custom data)
     */
    public function read(IngestConfig $config, mixed $payload = null): Generator;

    /**
     * Return the total number of rows, or null if unknown.
     */
    public function getTotalRows(): ?int;

    /**
     * Return the path where the processed file was stored, or null.
     */
    public function getProcessedFilePath(): ?string;

    /**
     * Clean up any temporary files or resources.
     */
    public function cleanup(): void;
}

# Tutorial: Building a JSON Array Handler

Let's build a handler that accepts a raw JSON array string. This is useful for importing data directly from a frontend POST request body.

# 1. Implement the Interface

Create app/Ingest/Handlers/JsonArrayHandler.php:

namespace App\Ingest\Handlers;

use Generator;
use InvalidArgumentException;
use LaravelIngest\Contracts\SourceHandler;
use LaravelIngest\IngestConfig;

class JsonArrayHandler implements SourceHandler
{
    protected ?int $count = null;

    public function read(IngestConfig $config, mixed $payload = null): Generator
    {
        // $payload will be the raw JSON string passed to IngestManager::start()
        if (!is_string($payload)) {
            throw new InvalidArgumentException("Payload must be a JSON string");
        }

        $data = json_decode($payload, true);

        if (!is_array($data)) {
            throw new InvalidArgumentException("Invalid JSON provided");
        }

        $this->count = count($data);

        // Yield each item. The framework handles the rest.
        foreach ($data as $item) {
            yield $item;
        }
    }

    public function getTotalRows(): ?int
    {
        return $this->count;
    }

    public function getProcessedFilePath(): ?string
    {
        return null; // We don't save a file to disk
    }

    public function cleanup(): void
    {
        // No file cleanup needed
    }
}

# 2. Register the Handler

Add it to config/ingest.php by mapping an existing SourceType enum value to your handler:

// config/ingest.php
'handlers' => [
    'upload' => LaravelIngest\Sources\UploadHandler::class,
    'filesystem' => LaravelIngest\Sources\FilesystemHandler::class,
    'ftp' => LaravelIngest\Sources\RemoteDiskHandler::class,
    'sftp' => LaravelIngest\Sources\RemoteDiskHandler::class,
    'url' => LaravelIngest\Sources\UrlHandler::class,
    
    // Map an existing SourceType to your custom handler
    // Option 1: Override an unused type
    // 'sftp' => App\Ingest\Handlers\JsonArrayHandler::class,
],

# 3. Alternative: Direct Handler Injection

For more flexibility, you can bypass the SourceType enum entirely by using the handler directly in your code:

use App\Ingest\Handlers\JsonArrayHandler;
use LaravelIngest\IngestManager;

// In a controller or service
$json = '[{"name": "Item 1", "email": "item1@example.com"}, {"name": "Item 2", "email": "item2@example.com"}]';

// Start the import with the custom handler
$ingestManager = app(IngestManager::class);
$run = $ingestManager->start('my-importer', $json);

# Example: API Response Handler

Here's a more practical example that fetches data from an external API:

namespace App\Ingest\Handlers;

use Generator;
use Illuminate\Support\Facades\Http;
use LaravelIngest\Contracts\SourceHandler;
use LaravelIngest\IngestConfig;
use LaravelIngest\Exceptions\SourceException;

class ApiHandler implements SourceHandler
{
    protected ?int $count = null;

    public function read(IngestConfig $config, mixed $payload = null): Generator
    {
        $url = $config->sourceOptions['url'] ?? null;
        $headers = $config->sourceOptions['headers'] ?? [];

        if (!$url) {
            throw new SourceException("API URL is required in source options");
        }

        $response = Http::withHeaders($headers)->get($url);

        if (!$response->successful()) {
            throw new SourceException("API request failed: " . $response->status());
        }

        $data = $response->json('data') ?? $response->json();
        
        if (!is_array($data)) {
            throw new SourceException("API response must contain an array");
        }

        $this->count = count($data);

        foreach ($data as $item) {
            yield $item;
        }
    }

    public function getTotalRows(): ?int
    {
        return $this->count;
    }

    public function getProcessedFilePath(): ?string
    {
        return null;
    }

    public function cleanup(): void
    {
        // Nothing to clean up
    }
}