incremental `docker image save <images> | xz -zc - >images.tar.xz` -


we have docker-compose project including various services, of share common base-images. after building images, 1 of our build job's post-build-steps docker image save <images> | xz -zc - >images.tar.xz create single compressed archive of images – used in offline-deployment fallback-strategy (so can transport these images via usb- or cd-media rather docker-registry). uncompressed docker image save <images> tar-stream 2gb in size. after piping through xz, compressed images.tar.xz 500mb.

this build job run often, , of time few images have changed. however, aforementioned docker … | xz … pipeline recreate images.tar.xz in entirety, requires time in overall build job. i'd optimize that.

is there way speed incremental builds?

i thought docker image save <imagen> | xz -zc - >imagen.tar.xz each image individually, can save modified images, result in twice required storage, because docker image save include duplicate base images between individual calls.

i able use single docker image save <images> invocation, update or re-compress actual changes in previous images.tar.xz. know that, because of how tar.xz structured, small changes – @ beginning of stream – require recreate whole file non-the-less. however, i'd gladly see solution involves splitting tar stream reasonably, such individual parts can updated.

note: aside meta/manifest files @ end, tar-stream contains bunch of layer folders, each of contains layer.tar , meta files, corresponding (de-duplicated) layers of saved images, e.g.:

$ xz -dc images.tar.xz | tar t 0166389787802d9a6c19a832fcfe976c30144d2430e798785110d8e8e562dab6/ 0166389787802d9a6c19a832fcfe976c30144d2430e798785110d8e8e562dab6/version 0166389787802d9a6c19a832fcfe976c30144d2430e798785110d8e8e562dab6/json 0166389787802d9a6c19a832fcfe976c30144d2430e798785110d8e8e562dab6/layer.tar ...(~100x4)... fa498ee40da8c70be99b8f451813d386b45da891353d7184cdb8dd1b40efca03/ fa498ee40da8c70be99b8f451813d386b45da891353d7184cdb8dd1b40efca03/version fa498ee40da8c70be99b8f451813d386b45da891353d7184cdb8dd1b40efca03/json fa498ee40da8c70be99b8f451813d386b45da891353d7184cdb8dd1b40efca03/layer.tar ffb2e673ba3e63b6b5922a482783b072759f0b83335a5ffab0b36dc804a24b93/ ffb2e673ba3e63b6b5922a482783b072759f0b83335a5ffab0b36dc804a24b93/version ffb2e673ba3e63b6b5922a482783b072759f0b83335a5ffab0b36dc804a24b93/json ffb2e673ba3e63b6b5922a482783b072759f0b83335a5ffab0b36dc804a24b93/layer.tar manifest.json repositories

ps: i'm using pxz instead of xz utilize cpu cores during compression, still takes considerable amount of time.


Comments

Post a Comment

Popular posts from this blog

Command prompt result in label. Python 2.7 -

i want put result(output) of shell command label. command works, label shows "0", however, in command prompt, result shown correctly, need showed in label. doing ubuntu. myg1 = button(root, text="rodyti informacija", command=lambda: gauti()) myg1.pack(side=bottom) def gauti(): imti = tekstas.get("1.0", "end-1c") info = subprocess.call("id '{imti}' ".format(imti=imti), shell=true) w = label(root, text= info) w.pack(side = bottom) subprocess.call() returns exit code of process created (in case 0 success). if want text output of process, should instead call subprocess.check_output() .
Read more

javascript - How do I use URL parameters to change link href on page? -

i change href value of link using url parameters. example: default <a id="dlink" href="link-1">link</a> but, when user goes https://url.com/?dlink=link-2 it swaps out link new link <a id="dlink" href="link-2">link</a> first need value url. if you're not using javascript framework can create function using jquery. please check get url parameter jquery or how query string values in js it. after that, element "a" id , set value there. $("#dlink").attr("href", hreffromparam);
Read more

amazon web services - AWS Route53 Trying To Get Site To Resolve To www -

so have domain nameservers pointed aws route 53. site hosted on ec2 instance , reachable resolves site.com, if input www.site.com. i'm curious how site resolve www.site.com. have under route53 domains hosted zone 2 additional record sets outside of ns , soa: a site.com ec2 elastic ip cname www.site.com ec2 public dns i can reach site using both site.com , www.site.com, resolve site.com. there way make if enter either, resolves www.site.com? at first, both should record set. a site.com ec2 elastic ip www.site.com ec2 elastic ip and need redirect setting in web server (nginx, apache, ..) following. (source: how redirect www non-www nginx on centos 7 ) redirect non-www www if want redirect users plain, non-www domain www domain, add server block: server { server_name example.com; return 301 $scheme://www.example.com$request_uri; } save , exit. configures nginx redirect requests "example.com" "www.example.com". not
Read more