city made out of glass : : close shot : : 3 5 mm, realism, octane render, 8 k, exploration, cinematic, trending on artstation, realistic, 3 5 mm camera, unreal engine, hyper detailed, photo – realistic maximum detail, volumetric light, moody cinematic epic concept art, realistic matte painting, hyper photorealistic, concept art, volumetric light, cinematic epic, octane render, 8 k, corona render, movie concept art, octane render, 8 k, corona render, cinematic, trending on artstation, movie concept art, cinematic composition, ultra – detailed, realistic, hyper – realistic, volumetric lighting, 8 k
city made out of glass : : close shot : : 3 5 mm, realism, octane render, 8 k, exploration, cinematic, trending on artstation, realistic, 3 5 mm camera, unreal engine, hyper detailed, photo – realistic maximum detail, volumetric light, moody cinematic epic concept art, realistic matte painting, hyper photorealistic, concept art, volumetric light, cinematic epic, octane render, 8 k, corona render, movie concept art, octane render, 8 k, corona render, cinematic, trending on artstation, movie concept art, cinematic composition, ultra – detailed, realistic, hyper – realistic, volumetric lighting, 8 k
n our discussions, I have noticed that we use the term "cold-storage" ambiguously.
Some of us think that "persistence" is "cold storage". But is not.
So, here, I define a cold-storage as any data source that we can use to restore data when we:
a. Start a new service and need to pre-populate Redis with some data
b. Have completely lost Redis with their persistence layer and needed to restore data from some data source (actually the same as point a.)
From such a perspective, we can use for cold-storage any of our data sources:
Most of our DBs (Clickhouse, Postgres, Vertica, etc)
Kafka
So, relative to Hot Data Store all our data storeges can be considered as Cold Storage.
A more important question here is: how should we schedule or run such a task for rebuilding HostDataStore from cold storage?
Here we have many options:
The main process can do it on startup
In my opinion, this is a preferable option. Because often the main task should know some metadata about the state of Hot Data Storage, and this option naturally provide it.
Also, in this case, we have one codebase for the main task and Hot Data Storage rebuilding task. And there is no added complexity in scheduling another rebuilding task.
Disadvantages:
it can be implemented only in Java tasks.
Hard to standardize
A distinct job in GitLab/ArgoCD that can deploy a special task to Flink to populate Hot Data Store.
Some imaginations:
We can standardize it:
We can store data in cold storage in parallel with Hot Data Storage. When the main task restarts, it can restore hot storage from cold storage.
Don't think is much different from a simple backup.
Long shot, woman posing nude, curvy hips, large breasts, realistic portrait, highly detailed, digital painting, artstation, concept art, smooth, sharp focus, illustration, cinematic lighting, art by artgerm and greg rutkowski and alphonse mucha
Ilya kuvshinov with long sky blue hair, gold eyes, amber eyes, boyish face, professional digital painting, wild brush strokes, concept art, award - winning photography, cinematic, wlop, color block, pop, hip, art by andy warhol, pixiv art, yoshitaka amano
beautiful fashion elegant goddness of water, chic strapless dress, tropical sea background, character design, in the style of artgerm, and wlop, chanel jewelry, cinematic lighting, hyperdetailed, 8 k realistic, symmetrical, global illumination, radiant light, love and mercy, frostbite 3 engine, cryengine, dof, trending on artstation, digital art, crepuscular ray
Long shot, woman posing nude, curvy hips, large breasts, realistic portrait, highly detailed, digital painting, artstation, concept art, smooth, sharp focus, illustration, cinematic lighting, art by artgerm and greg rutkowski and alphonse mucha
Long shot, woman posing nude, curvy hips, large breasts, realistic portrait, highly detailed, digital painting, artstation, concept art, smooth, sharp focus, illustration, cinematic lighting, art by artgerm and greg rutkowski and alphonse mucha
ornate decaying glass orb house, heavy rain, spooky night scene, phosphorescence, tasmanian wilderness, by ansell adams and peter dombrovskis,ornate, poetic, 3d render, digital art, octane render, 8k artistic photography, photo-realistic
