Skip to content

CSV Parsing Tests

Ticket: Add unit tests for CSV parsing robustness
Type: Enhancement / Quality
Affected Component: code_source_simule/pipeline.py, tests/test_pipeline.py

1. Context

The read_and_clean_csv function is critical to ensuring that incoming data is correct. It is used by the ETL pipeline to normalize CSV files and trigger price enrichment.

2. Objective

Strengthen the unit test suite so the function remains robust against degraded inputs, without changing its existing behavior.

3. Additions made

  • Added unit tests covering:
  • a CSV missing the Ticker column (should return an empty DataFrame)
  • invalid Price values (N/A, format 1 234.56) that should be normalized to a float
  • missing Marketstack_Currency column (should be added and set to USD)
  • automatic filtering of Cash rows and rows with missing tickers

4. Result

The pytest suite now passes successfully, and the new scenarios strengthen coverage of the CSV cleaning function.