LightOJ (https://lightoj.com) is a Bangladeshi online platform to learn programming and algorithmic problem solving. It was created around 2011 by Jane Alam Jan. This platform helped many Bangladeshi problem solvers to learn and improve algorithmic problem solving skills.
The system which was developed around 2011 was serving well, but also adding some problems. For example, it was not designed for scalability in mind, db queries became slower, UI became old and doesn’t go with the current trends.
So Jane Alam Jan vai, took initiative in late 2017 to build a new and better version of LightOJ, which will use new…
Google BigQuery is one of the more advanced data warehouses in the market, and has out of the box support for building and training ML models using SQL like statements without requiring any code. This is extremely powerful, however managing end to end ML pipelines in this way can be fragile and requires manual steps to updating training and prediction.
In this article we walk through building a simple end to end BigQuery ML pipeline using the open-source framework Dataform to help us manage the end to end process of data preparation, training and prediction.
Every software developer knows (should know) Github, its a cloud hosting provider for git repository. As of January 2020, Github has over 40 million users and more than 100 million repositories. I have started using Github since March 8, 2012, and after started using it my developer life has changed a lot. My progression as a programmer or developer is influenced a lot by Github. Today I will describe how Github is helping (or helped) me to improve my software development skillset.
Github is the number one choice for hosting open source projects, almost all the popular open source projects…
Having a state management library on a React based single page application is quite useful, especially if the application is complex in nature, for example, if we want to share states between two react components which are neither siblings nor child. But even if you use a state management library, it might not solve the application state in a clean and expected way.
We initially used our in-house developed state management tool, which I will refer to as Goggle Store in this whole article. Goggle Store follows object oriented style, where you need to create state entity and state entities…
Big data is a hot topic now. People are mostly focused on machine learning, data analysis, report generation etc. That’s why you can find many online courses to learn more about data science and data analysis. But when it comes to make data useful to the company, it requires not only data scientists or data analysts but also data engineers. And my goal of this article is to give some introduction about data engineering role to the fresh grads or anyone interested in this role.
There is no straightforward definition. One can say data engineer is a software engineer who…
If you don’t know what is competitive programming, please check this Wikipedia article. Basically, competitive programming is a programming contest where participants try to solve some well-defined problems (not like hackathon) by coding efficient algorithms in general.
In this article, I am going to talk about how competitive programming helped me to become a better software engineer and how I had to adapt during the transition from competitive programming to software engineering.
I was a regular competitive programmer during the period 2011–2016, when I participated in many national, international level online and on-site contests. My achievements are not quite big…
On my previous article I have described about the application itself and how to deploy it on Kubernetes. But it will not work for large number of users and also that application was not fault tolerant. Lets discuss how we can horizontally scale that application on Kubernetes in this article.
All the codes are available here: https://github.com/faiyaz26/rtn-k8s
Anyone who is familiar with kubernetes and horizontal scaling will think about increasing the replica size of the pods. Lets try that first:
kubectl scale deployment rtn-k8s-deployment --replicas=3
Now-a-days real time notification feature is quite popular among various kind of web applications. This article will go through the process of building a real time notification system, deploying it on Kubernetes and scale it for thousands of users. I assume that you already know about Node.JS, Docker and Kubernetes, because I will use these tools for this post and there will be no in depth explanations. And also please note that I will just show the basic implementation which is not production ready ( security, code quality, testing) but can serve thousands of users. :)
Have you ever heard of SPOJ, CodeForces, Hackerrank, UVa ? If you are in competitive programming line, you might have know these sites. They are called online judge. What is an online judge by the way ?
An online judge is an online system to test programs in programming contests. They are also used to practice for such contests. Many of these systems organize their own contests.
The system can compile and execute code, and test them with pre-constructed data. Submitted code may be run with restrictions, including time limit, memory limit, security restriction and so on. The…
A noob software engineer