Skip to content

Weekend Logic Adjustment

Ticket #72: Adjustment of the Data Pipeline's Weekend Logic
Type: Enhancement / Maintenance
Affected Component: code_source_simule/pipeline.py, Cron Configuration

1. Context and Strategic Objective

The objective was to ensure a comprehensive capture of the week's market data while optimizing resource usage. It was necessary to ensure that Friday's closing data was correctly retrieved, but that the system would not attempt unnecessary executions when the markets are closed (Saturday and Sunday).

2. Investigation Process

The starting point was the observation that there was a potential flaw in the weekend logic: Friday's data appeared to be missing.

  1. Initial Hypothesis: The first hypothesis was that the pipeline.py script contained a logical error that prevented its execution on Saturday.

  2. Code Analysis: An inspection of the pipeline.py file was conducted. The code contained the following condition: 

    # today.weekday() returns 0 for Monday, 5 for Saturday, 6 for Sunday.
if today.weekday() in [6, 0]: # 6=Sunday, 0=Monday
    print(f"Pipeline not executed ({today.strftime('%A')}). No market data for the previous day (weekend).")
    return None

  3. Discovery of a Contradiction: The code analysis proved that the script was, in fact, correct. It correctly stopped on Sunday (day 6) and Monday (day 0), but should have run without issue on Saturday (day 5). However, the problem persisted.

  4. New Hypothesis: The contradiction between correct code and erroneous system behavior led to a new hypothesis. The problem was not with the script itself, but with its orchestrator. The most likely hypothesis was that the cron task on the server was not configured to trigger on Saturday.

3. Root Cause Identified

The root cause was not a bug in the application code, but an infrastructure configuration error. The server's crontab was configured to run the script only from Monday to Friday (e.g., * * * * 1-5), thus omitting the necessary Saturday execution.

4. Implemented Solution

The solution required no modification to the Python code. The fix was applied directly at the infrastructure level by modifying the crontab configuration to include Saturday in the execution days.

5. Justification and Benefits

This investigation highlights a fundamental engineering principle: always investigate the entire chain. By not stopping at the first hypothesis and by analyzing the code, we avoided unnecessarily complicating a script that was already correct. The precise diagnosis allowed targeting the true root cause, leading to a simple, effective solution with no impact on the codebase.