job_heartbeat_sec = 4 # The scheduler constantly tries to trigger new tasks (look at the # scheduler section in the docs for more information).
Scheduler_zombie_task_threshold = 120 # Task instances listen for external kill signal (when you clear tasks # from the CLI or the UI), this defines the frequency at which they should # listen (in seconds). # scheduler_zombie_task_threshold should be higher then # This is used by the health check in the "/health" endpoint # ago (in seconds), scheduler is considered unhealthy. If the last scheduler heartbeat happened more than scheduler_health_check_threshold #consider change the heart rate scheduling of scheduler # This defines how many threads will run. # The scheduler can run multiple threads in parallel to schedule dags. # How long before timing out a DagFileProcessor, which processes a dag file # default is 30, i suggest 3000 for dynamic dag operators. # How long before timing out a python file import # The maximum number of active DAG runs per DAG # i suggest double the defaults after installation. # The number of task instances allowed to run concurrently by the scheduler # the max number of task instances that should run simultaneously # no limit will be placed on the total number of concurrent connections. # max_overflow can be set to -1 to indicate no overflow limit Be sure to understand what your are doing. Usually the config file is located in ~/airflow/airflow.cfg Notice this configuration is provided AS IS. When you are using dynamic operators, the default settings will not work.I suggest the below settings.
0 kommentar(er)
