Parameters ( dict or iterable) - The parameters to render the SQL query with (optional).
That contains a non-zero / empty string value. To pass, it needs to return at least one cell A fail_on_empty boolean can alsoīe passed to the sensor in which case it will fail if no rows have been returned ParametersĬonn_id ( str) - The connection to run the sensor against If failure callable is defined and the criteria is met the sensor will raise AirflowException.įailure criteria is evaluated before success criteria. If success callable is defined the sensor will keep retrying until the criteria is met. Optional success and failure callables are called with the first cell returned as the argument. Success or failure criteria are met, or if the first cell is not in (0, '0', '', None). Runs a sql statement repeatedly until a criteria is met.
#AIRFLOW 2.0 FULL#
The full changelog is about 3,000 lines long (already excluding everything backported to 1.10), so for now I’ll simply share some of the major features in 2.0.0 compared to 1.10. maxactivetasksperdag 1000 <- enables running 1000 tasks in parallel for one DAG. I am proud to announce that Apache Airflow 2.0.0 has been released. SqlSensor ( *, conn_id, sql, parameters = None, success = None, failure = None, fail_on_empty = False, ** kwargs ) ¶īases: workerconcurrency 36 <- this variable states how many tasks can be run in parallel on one worker (in this case 28 workers will be used, so we need 36 parallel tasks 28 36 1008) parallelism 1000 <- enables running 1000 tasks in parallel.
0 kommentar(er)
