Introduction There are lots of benefit of using MySQL as a backend database in Airflow. Main reason is that MySQL is widely used in production instead of SQlite. Also, we can have scalable database system where we can have concurrent requests, high security and well defined permission and roles. From the last few blogs, I’ve […]
Search Results for: airflow
Branching Tasks in Airflow For Dynamic Flow of DAGs
Branching Task in Airflow When do we need to make a branch like flow of a task? A simple example could be, lets assume that we are in a Media Company and our task is to provide personalized content experience. Lets assume that we will have 3 different sets of rules for 3 different types […]
Airflow in WSL and Dynamic Tasks
This blog is a continuation of previous blog Getting Started With Airflow in WSL. Dynamic Tasks in Airflow Sometimes there will be a need to create different task for different purpose within a DAG and those task has to be run dynamically. Not only run but has to be created dynamically also. A simple example […]
Running Airflow In WSL and Getting Started With It
Airflow in WSL: Getting Started With I might stop to write new blogs in this site so please visit dataqoil.com for more cool stuffs. Blog Versions This blog will be updated frequently. 2021-12-01 Written blog. 2021-12-05 Updated contents upto TaskFlowAPI. 2021-12-07 Updated contents upto SubDAGs. 2021-12-13 Updated contents upto TaskGroup. 2022-01-09 Started dynamic tasks in […]
Making Money with Programming
This post has been cross-posted from q-viper.github.io. How much do you know about Making money with programming in this decade? It is getting broader and more expansive. With the availability of a wide range of internet and lots of technology, many people are engaged with technology nowadays and the need to provide good tech has […]
Weather Bot with Twilio
To make a Weather Bot we are going to use Twilio, it is a platform which allows us to send SMS, make calls using methods like API calls. It was the beginning of COVID crisis and I was living in my village where internet was available by only cellular 3G data. But it is a […]
Apache Superset for COVID Dashboards
Apache Superset is a very useful and easy-to-use visualization and dashboard-making tool that can be an alternative to tools like Tableau and PowerBI. In this blog, we will explore how we can create awesome data dashboards using Apache superset with little to no code at all. But there are a few things one should do […]
Connecting to MySQL Server in Windows Machine from WSL
Connecting MySQL Server in Windows Machine from WSL What does this mean? In simple sentence, how do we connect to a MySQL server which is hosted in Windows from WSL. It might sound easy but let me tell you, IT IS NOT!!!! I was trying to connect (from WSL) to my local MySQL which was […]