Folder Mode

Folder mode processes CSV files from a folder using a mailbox-style structure.

Run folder mode by passing a directory as <path>:

optimus check ./inbox --map mapping.yml --standard

Folder Structure

The CLI creates these folders inside the folder you pass:

inbox/
  processing/
  done/
  error/
  results/

Files are picked up from the root folder only, not from subdirectories.

In --dry-run, folder mode reads matching files from the root folder in place. It does not create the mailbox folders, retry files from error, move files, write result or error files, clean up done, or create a support log.

Processing Flow

1. The CLI creates processing, done, error, and results if they do not exist.
2. If --retry or folder.retry: true is set, failed CSV files from error are moved back to the root folder for another attempt.
3. The CLI selects files from the root folder using pattern.
4. Result files ending in _result are ignored.
5. Files younger than minAge seconds are ignored.
6. Each selected file is moved to processing.
7. The file is validated.
8. The result CSV is written to results.
9. Successful files and validation-error files move to done.
10. CLI, mapping, authentication, IO, temporary, or quota errors move the source file to error and create an .err file.

Folder Options

Command-line options:

optimus check ./inbox --map mapping.yml --standard --pattern "*.csv" --min-age 30

Retry files that previously moved to error:

optimus check ./inbox --map mapping.yml --standard --retry

YAML options:

folder:
  pattern: "*.csv"
  minAge: 30
  onDuplicate: skip
  retry: false
  cleanupDoneAfterDays: 30

Key	Default	Description
pattern	*.csv	File pattern selected from the folder root.
minAge	0	Minimum file age in seconds.
onDuplicate	skip	What to do when moving a file would collide with an existing file in done, error, or the folder root during retry. See duplicate handling below.
retry	false	Move failed CSV files from error back to the root folder for another attempt. Retried files start a new validation operation and replace their saved state.
cleanupDoneAfterDays	not set	Delete old files and saved state from done.

Retrying Failed Files

When a file fails because of a CLI, mapping, authentication, IO, temporary, or quota error, the CLI moves the source CSV to error and writes an .err file.

After you fix the cause, run folder mode with --retry or set folder.retry: true. The CLI moves matching CSV files from error back to the root folder, removes the .err marker, and processes them again.

Retry starts a new validation operation. It does not cancel any older remote operation.

Duplicate Handling

onDuplicate controls what happens when the CLI tries to move a CSV file and a file with the same name already exists at the destination:

Value	Behavior
skip	Keep the existing destination file. The current processed file stays in processing for manual review. During --retry, the failed file stays in error.
overwrite	Replace the existing destination file and its .optimus-state.json sidecar.
rename	Keep both files by adding a timestamp to the file being moved.

Result And Error Files

For an input file named customers.csv, folder mode writes:

results/customers_result.csv

If a file cannot be processed, the CLI writes an error file:

error/customers.err

The .err file includes the source file name, exit code, timestamp, support log path when available, and a short error description.