Artificial intelligence is now part of our everyday tools and technology. Ability of the machines to learn and reason like human is groundbreaking. At the heart of the machine intelligence is the AI models. GPU (Graphics Processing Unit) is the main component to process large amount of data and build AI models. Jupyter Notebook is […]
GPU enabled JupyterHub on GKE Autopilot Read More »
Apache Airflow supports multiple database backends; PostgreSQL, MySQL and SQLite. However, SQLite is only recommended for the development purposes due to many limitations. That leaves PostgreSQL and MySQL as two options for Apache Airflow database backend for a production environment. When I started working on a framework using Apache Airflow, I chose MySQL as a
Apache Airflow Database Backend MySQL vs PostgreSQL Read More »
Apache Airflow is the most widely used workflow management platform. I have been building a workflow system using Apache Airflow for more than a year now. The system is required to support workflows that handle large amount of data and scale on-demand. Even though there is help and documentation available online, I ran into few
Apache Airflow optimizations Read More »
I started working with JupyterHub on Google Kubernetes Engine using zero-to-jupyterhub couple of months ago. During this time, I have not only resolved multiple issues but also have documented various troubleshooting techniques. JupyterHub has many features and zero-to-jupyterhub provides extensive configuration capabilities to use them. One of the most basic features required for the most
GCS bucket with JupyterHub on GKE Read More »
Cilium is an open source, cloud native solution for providing, securing, and observing network connectivity between workloads, fueled by the revolutionary Kernel technology eBPF. It is mainly used for transparently securing the network connectivity between application services deployed using Linux container management platforms like Docker and Kubernetes. All major cloud providers are working to adopt
Troubleshooting Cilium on Google Kubernetes Engine Read More »
Jupyter is a well-known open-source project that develops and supports interactive computing products such as Jupyter notebook, JupyterLab and JupyterHub. These products are widely used by data science communities world wide. JupyterHub is a multi-user Jupyter notebook environment. It allows users to develop web-based interactive notebook documents without burdening them with installation and maintenance. JupyterHub
JupyterHub on GKE Autopilot Read More »
Software containerization is the modern application programming model that has revolutionized the present day enterprise application development. It not only helps to make the application portable but also makes it scalable, I recently added a post that describes building GraphQL service as a containerized application. The main component of a containerized application is the orchestration
Google Kubernetes Engine Autopilot Read More »