Warning: this article follows an Ariadne’s thread, that is, all the links present. It is therefore difficult to understand it as an independent piece, unless you already have the preparation because you are part of the insiders.
As vaguely mentioned in the previous article there are several ways to deliver a statistical service. In that case I had all the ingredients on a local PC, but there are other ways.
Container = frozen food, just throw it in the pan, the manufacturer has already thought of the ingredients, recipe, packaging for the whole thing. Examples: ready-made lasagna.
Kitchen = hypervisor, virtual machines (VM). A fully equipped kitchen, where you have access to all the fresh ingredients and utensils needed to prepare a meal from scratch. In this case, you are the chef (or system administrator), and have complete control over how to combine the ingredients and prepare the dish to your specific needs. Hypervisor-based virtualization gives you flexibility and complete control over the configuration and use of resources, but it also requires more work and planning on your part than simply heating up (starting) a ready meal like you would with a container. The hypervisor allows for different kitchens (operating system), for example, one that makes sushi, one that makes Mexican, one that makes kebab, one that makes pizza. The hypervisor allows various virtual machines to be monitored, just as the chef monitors and coordinates the cooks in the kitchen, which can have various areas of operation: cutting and preparing meats, slicing and preparing fish, cleaning and preparing vegetables, preparing appetizers and cold dishes, preparing desserts, pastry department, etc. In a YouTube video you see that I programmatically prepared a kitchen, provided by Microsoft, also specifying the utensils and ingredients (program, libraries).
At the level of infrastructure the two solutions differ, but we are not interested in that here; however, both have virtualization, or abstraction, in common.
The former abstracts “dishes,” the latter abstracts what comes long before a dish: pans, frying pans, oven, chopping, etc. The hypervisor virtualizes hardware, so we can say it creates computers within a computer; docker containers abstract everything above the operating system, so it sits at a higher level.
Containers allow for portable solutions, just like dishes; a waiter can carry a maximum of 6, depending on his physical/acrobatic performance. Similarly, when containers require more than a single computer, you need Kubernetes (a kind of head waiter, room manager).
Of course, you can’t make a metaphor about all aspects of this comparison. This article may seem like an excessively long and technical tour to understand the more computer-based part of statistics, but ignoring it can make statistical services for your company less efficient from a cash-flow perspective.
At this point you will wonder when you need the ready-made or frozen dish and when you need the kitchen. It depends essentially on the courses your business needs. Certain statistical services encompass many sub-services. Or rather, certain domains of statistics, like data engineering, have many services. In the case of Airbyte, an application with significant computational demands, you need a dedicated virtual machine.
For a home-hosted dashboard (e.g. Shiny), or a statistical model that makes 20 predictions a day or less, you only need two containers, possibly running in very cheap virtual machines or a single cheap one.
Containers are best when isolation is also needed, which would deserve a separate article.
There is another approach to choosing kitchen or ready-made dish: if the application already has an image on docker, then it is better to go there. Most classic example: the postgreSQL database. Although some argue that having a containerized database may give more problems.
So in most cases it is convenient a container for a micro and small company, because SMEs generally ask for more microservices (dishes) than infrastructures (kitchen). From medium-sized companies onwards the discussion changes. If you simply want to eat, you do not need to also buy the kitchen as in the case of hypervisors / virtual machines, also because it costs more in terms of resources, you can buy a ready-made dish, where someone has already taken care of the study of the dish.
In conclusion, I would like to point out that the metaphor of the kitchen in other Enterprise Statistics contents has another meaning.
Do you have concerns about how all this talk has to do with your company’s data implantation? Let’s hear from you in a free call.