[{"data":1,"prerenderedAt":1115},["ShallowReactive",2],{"article-\u002Farticles\u002Fk8s-rolling-update-zero-downtime":3},{"id":4,"title":5,"body":6,"date":1101,"description":1102,"excerpt":1103,"extension":1104,"meta":1105,"navigation":608,"path":1106,"seo":1107,"stem":1108,"tags":1109,"__hash__":1114},"articles\u002Farticles\u002Fk8s-rolling-update-zero-downtime.md","Kubernetes 滾動更新怎麼做到真的不停機？maxSurge、Readiness、preStop 與 PodDisruptionBudget",{"type":7,"value":8,"toc":1086},"minimark",[9,23,26,48,52,59,62,89,96,99,205,219,226,231,242,256,314,321,325,331,337,344,368,375,379,389,471,474,477,500,503,506,529,539,667,682,696,700,707,735,739,750,761,845,848,851,888,891,901,921,925,928,978,1024,1027,1030,1073,1076,1082],[10,11,12,13,18,19],"p",{},"Pod、Deployment、Service 的基本關係可以先參考 ",[14,15,17],"a",{"href":16},"\u002Farticles\u002Fk8s-core-objects","Kubernetes 入門","。這篇要回答的問題是：",[20,21,22],"strong",{},"Deployment 明明就是滾動更新，為什麼每次部署還是有一小段錯誤？",[10,24,25],{},"答案是，「不停機」需要三段都顧到：",[27,28,29,36,42],"ol",{},[30,31,32,35],"li",{},[20,33,34],{},"新 Pod","：真的準備好了才接流量",[30,37,38,41],{},[20,39,40],{},"舊 Pod","：確定沒有流量了才關閉，關閉前把手上的請求處理完",[30,43,44,47],{},[20,45,46],{},"其他原因的中斷","：Node 維護、叢集升級時，不能一次把 Pod 全部趕走",[49,50,51],"h2",{"id":51},"滾動更新的流程",[10,53,54],{},[55,56],"img",{"alt":57,"src":58},"Kubernetes 滾動更新流程，3 個副本、maxSurge 1、maxUnavailable 0：第一步先建立一個 v2 Pod，此時共有 4 個 Pod，v2 還沒通過 Readiness 不接流量；第二步 v2 Pod 就緒後加入 Service，再終止一個 v1 Pod；重複這個過程，第三步、第四步依序替換，最後 3 個 Pod 全部是 v2；整個過程中可接流量的 Pod 從未少於 3 個","\u002Fimages\u002Farticles\u002Fk8s-rolling-update.svg",[10,60,61],{},"修改 Deployment 的映像檔版本後，Deployment 會建立一個新的 ReplicaSet，然後依兩個參數逐步替換：",[63,64,65,78],"ul",{},[30,66,67,73,74,77],{},[20,68,69],{},[70,71,72],"code",{},"maxSurge","：更新過程中，最多可以",[20,75,76],{},"比期望數量多幾個"," Pod",[30,79,80,85,86],{},[20,81,82],{},[70,83,84],{},"maxUnavailable","：更新過程中，最多可以有",[20,87,88],{},"幾個 Pod 不可用",[10,90,91,92,95],{},"預設都是 ",[70,93,94],{},"25%","。以 4 個副本為例，代表最多可以多開 1 個、同時可以有 1 個不可用。",[10,97,98],{},"對線上服務，我會這樣設：",[100,101,106],"pre",{"className":102,"code":103,"language":104,"meta":105,"style":105},"language-yaml shiki shiki-themes github-dark","spec:\n  replicas: 3\n  strategy:\n    type: RollingUpdate\n    rollingUpdate:\n      maxSurge: 1          # 每次多開一個新的\n      maxUnavailable: 0    # 新的就緒之前，舊的一個都不關\n  minReadySeconds: 10      # 就緒後再觀察 10 秒，才算可用\n","yaml","",[70,107,108,121,134,142,154,162,177,191],{"__ignoreMap":105},[109,110,113,117],"span",{"class":111,"line":112},"line",1,[109,114,116],{"class":115},"s4JwU","spec",[109,118,120],{"class":119},"s95oV",":\n",[109,122,124,127,130],{"class":111,"line":123},2,[109,125,126],{"class":115},"  replicas",[109,128,129],{"class":119},": ",[109,131,133],{"class":132},"sDLfK","3\n",[109,135,137,140],{"class":111,"line":136},3,[109,138,139],{"class":115},"  strategy",[109,141,120],{"class":119},[109,143,145,148,150],{"class":111,"line":144},4,[109,146,147],{"class":115},"    type",[109,149,129],{"class":119},[109,151,153],{"class":152},"sU2Wk","RollingUpdate\n",[109,155,157,160],{"class":111,"line":156},5,[109,158,159],{"class":115},"    rollingUpdate",[109,161,120],{"class":119},[109,163,165,168,170,173],{"class":111,"line":164},6,[109,166,167],{"class":115},"      maxSurge",[109,169,129],{"class":119},[109,171,172],{"class":132},"1",[109,174,176],{"class":175},"sAwPA","          # 每次多開一個新的\n",[109,178,180,183,185,188],{"class":111,"line":179},7,[109,181,182],{"class":115},"      maxUnavailable",[109,184,129],{"class":119},[109,186,187],{"class":132},"0",[109,189,190],{"class":175},"    # 新的就緒之前，舊的一個都不關\n",[109,192,194,197,199,202],{"class":111,"line":193},8,[109,195,196],{"class":115},"  minReadySeconds",[109,198,129],{"class":119},[109,200,201],{"class":132},"10",[109,203,204],{"class":175},"      # 就緒後再觀察 10 秒，才算可用\n",[10,206,207,210,211,214,215,218],{},[70,208,209],{},"maxUnavailable: 0"," 代表",[20,212,213],{},"可接流量的 Pod 永遠不會少於 3 個","：先開一個新的，等它就緒，才關一個舊的。代價是更新期間需要多一個 Pod 的資源，叢集資源不夠時，新 Pod 會卡在 ",[70,216,217],{},"Pending","，整個更新就停住。",[10,220,221,222,225],{},"另一種策略 ",[70,223,224],{},"Recreate"," 是先全部關掉再全部開新的，一定會停機，只適合「新舊版本絕對不能同時存在」的情況。",[227,228,230],"h3",{"id":229},"readiness-probe判斷新-pod-準備好了","Readiness Probe：判斷「新 Pod 準備好了」",[10,232,233,234,237,238,241],{},"上面的流程有一個關鍵：「",[20,235,236],{},"新 Pod 就緒","」是怎麼判斷的？答案是 ",[20,239,240],{},"Readiness Probe","。",[63,243,244,250],{},[30,245,246,249],{},[20,247,248],{},"沒有設定 Readiness Probe","：容器一啟動就被視為就緒，Deployment 馬上關掉一個舊的；但 .NET 應用程式可能還在暖機、建立資料庫連線池，前幾秒的請求會變慢或失敗",[30,251,252,255],{},[20,253,254],{},"有設定","：要等探針通過，Pod 才會被加進 Service，Deployment 才會繼續下一步",[100,257,259],{"className":102,"code":258,"language":104,"meta":105,"style":105},"readinessProbe:\n  httpGet:\n    path: \u002Fhealth\u002Fready\n    port: 8080\n  periodSeconds: 5\n  failureThreshold: 3\n",[70,260,261,268,275,285,295,305],{"__ignoreMap":105},[109,262,263,266],{"class":111,"line":112},[109,264,265],{"class":115},"readinessProbe",[109,267,120],{"class":119},[109,269,270,273],{"class":111,"line":123},[109,271,272],{"class":115},"  httpGet",[109,274,120],{"class":119},[109,276,277,280,282],{"class":111,"line":136},[109,278,279],{"class":115},"    path",[109,281,129],{"class":119},[109,283,284],{"class":152},"\u002Fhealth\u002Fready\n",[109,286,287,290,292],{"class":111,"line":144},[109,288,289],{"class":115},"    port",[109,291,129],{"class":119},[109,293,294],{"class":132},"8080\n",[109,296,297,300,302],{"class":111,"line":156},[109,298,299],{"class":115},"  periodSeconds",[109,301,129],{"class":119},[109,303,304],{"class":132},"5\n",[109,306,307,310,312],{"class":111,"line":164},[109,308,309],{"class":115},"  failureThreshold",[109,311,129],{"class":119},[109,313,133],{"class":132},[10,315,316,317,320],{},"Readiness 端點應該檢查「這個實例能不能正常處理請求」，例如必要的設定與快取載入完成、資料庫連得上。新版本如果一直無法就緒，滾動更新會停在那裡，",[20,318,319],{},"舊版本的 Pod 還在","，服務不會中斷，這是 Readiness 帶來的另一個保護。",[49,322,324],{"id":323},"舊-pod-的關閉最容易被忽略的-502","舊 Pod 的關閉：最容易被忽略的 502",[10,326,327,328,241],{},"即使新 Pod 都正確就緒了，很多人還是會在部署時看到一小段 502。原因出在",[20,329,330],{},"舊 Pod 關閉的方式",[10,332,333],{},[55,334],{"alt":335,"src":336},"Pod 終止的時序比較：Pod 被標記為 Terminating 後，有兩件事同時開始。一條線是 Endpoints 控制器把 Pod 從 Service 移除，再由 kube-proxy 與 Ingress Controller 更新各自的轉發規則，這需要幾秒；另一條線是 kubelet 送出 SIGTERM。沒有 preStop 時，應用程式收到 SIGTERM 立刻停止接受連線，但轉發規則還沒更新，仍有請求被送進來，造成 502。加上 preStop sleep 10 秒後，SIGTERM 延後送出，這段時間轉發規則已經更新完成，應用程式接著處理完手上的請求再結束，整體時間要在 terminationGracePeriodSeconds 之內","\u002Fimages\u002Farticles\u002Fk8s-pod-termination.svg",[10,338,339,340,343],{},"一個 Pod 被刪除時，K8s 會",[20,341,342],{},"同時","做兩件事：",[27,345,346,355],{},[30,347,348,351,352],{},[20,349,350],{},"把 Pod 從 Service 移除","：Endpoints 更新後，各個 Node 上的 kube-proxy、Ingress Controller 要各自更新轉發規則，",[20,353,354],{},"這需要幾秒鐘",[30,356,357,360,361,364,365],{},[20,358,359],{},"通知容器關閉","：kubelet 執行 ",[70,362,363],{},"preStop","（如果有），然後送出 ",[20,366,367],{},"SIGTERM",[10,369,370,371,374],{},"問題就在「同時」。應用程式收到 SIGTERM 就停止接受新連線，但轉發規則可能還沒更新完，",[20,372,373],{},"還有請求被送到這個正在關閉的 Pod","，結果就是連線被拒絕、502。",[227,376,378],{"id":377},"prestop先等一下再關","preStop：先等一下再關",[10,380,381,382,384,385,388],{},"解法是用 ",[70,383,363],{}," 讓 SIGTERM ",[20,386,387],{},"晚幾秒再送","，給轉發規則足夠的時間更新：",[100,390,392],{"className":102,"code":391,"language":104,"meta":105,"style":105},"spec:\n  terminationGracePeriodSeconds: 45\n  containers:\n    - name: house-api\n      lifecycle:\n        preStop:\n          exec:\n            command: [\"sleep\", \"10\"]\n",[70,393,394,400,410,417,430,437,444,451],{"__ignoreMap":105},[109,395,396,398],{"class":111,"line":112},[109,397,116],{"class":115},[109,399,120],{"class":119},[109,401,402,405,407],{"class":111,"line":123},[109,403,404],{"class":115},"  terminationGracePeriodSeconds",[109,406,129],{"class":119},[109,408,409],{"class":132},"45\n",[109,411,412,415],{"class":111,"line":136},[109,413,414],{"class":115},"  containers",[109,416,120],{"class":119},[109,418,419,422,425,427],{"class":111,"line":144},[109,420,421],{"class":119},"    - ",[109,423,424],{"class":115},"name",[109,426,129],{"class":119},[109,428,429],{"class":152},"house-api\n",[109,431,432,435],{"class":111,"line":156},[109,433,434],{"class":115},"      lifecycle",[109,436,120],{"class":119},[109,438,439,442],{"class":111,"line":164},[109,440,441],{"class":115},"        preStop",[109,443,120],{"class":119},[109,445,446,449],{"class":111,"line":179},[109,447,448],{"class":115},"          exec",[109,450,120],{"class":119},[109,452,453,456,459,462,465,468],{"class":111,"line":193},[109,454,455],{"class":115},"            command",[109,457,458],{"class":119},": [",[109,460,461],{"class":152},"\"sleep\"",[109,463,464],{"class":119},", ",[109,466,467],{"class":152},"\"10\"",[109,469,470],{"class":119},"]\n",[10,472,473],{},"這 10 秒內 Pod 照常處理請求，但已經不會有新的請求進來了。10 秒後才送 SIGTERM，應用程式開始正常關閉。",[10,475,476],{},"要注意兩件事：",[63,478,479,488],{},[30,480,481,487],{},[20,482,483,486],{},[70,484,485],{},"terminationGracePeriodSeconds"," 包含 preStop 的時間","。預設 30 秒，時間到了不管做完沒有都會送 SIGKILL 強制結束。preStop 10 秒、應用程式關閉最多 30 秒，這裡就要設 40 秒以上",[30,489,490,496,497,499],{},[20,491,492,493],{},"映像檔裡要有 ",[70,494,495],{},"sleep","：精簡過的映像檔（例如 distroless、chiseled）沒有 shell，也沒有 ",[70,498,495],{}," 指令，preStop 會執行失敗。這種情況可以改在應用程式裡處理（見下一節）",[227,501,502],{"id":502},"應用程式端的優雅關閉",[10,504,505],{},"收到 SIGTERM 後，ASP.NET Core 會：",[27,507,508,511,518],{},[30,509,510],{},"停止接受新的連線",[30,512,513,514,517],{},"等待",[20,515,516],{},"正在處理中的請求","完成",[30,519,520,521,524,525,528],{},"呼叫所有 ",[70,522,523],{},"IHostedService"," 的 ",[70,526,527],{},"StopAsync","，讓背景工作收尾",[10,530,531,532,535,536,538],{},"等待的上限由 ",[70,533,534],{},"HostOptions.ShutdownTimeout"," 決定（.NET 6 之後預設 30 秒），要確保它加上 preStop 的時間，小於 ",[70,537,485],{},"：",[100,540,544],{"className":541,"code":542,"language":543,"meta":105,"style":105},"language-csharp shiki shiki-themes github-dark","builder.Services.Configure\u003CHostOptions>(options =>\n{\n    options.ShutdownTimeout = TimeSpan.FromSeconds(25);\n});\n\n\u002F\u002F 映像檔沒有 sleep 可用時，在應用程式裡延後關閉，效果等同 preStop\nvar app = builder.Build();\napp.Lifetime.ApplicationStopping.Register(() => Thread.Sleep(TimeSpan.FromSeconds(10)));\n","csharp",[70,545,546,571,576,599,604,610,615,635],{"__ignoreMap":105},[109,547,548,551,555,558,561,564,567],{"class":111,"line":112},[109,549,550],{"class":119},"builder.Services.",[109,552,554],{"class":553},"svObZ","Configure",[109,556,557],{"class":119},"\u003C",[109,559,560],{"class":553},"HostOptions",[109,562,563],{"class":119},">(",[109,565,566],{"class":553},"options",[109,568,570],{"class":569},"snl16"," =>\n",[109,572,573],{"class":111,"line":123},[109,574,575],{"class":119},"{\n",[109,577,578,581,584,587,590,593,596],{"class":111,"line":136},[109,579,580],{"class":119},"    options.ShutdownTimeout ",[109,582,583],{"class":569},"=",[109,585,586],{"class":119}," TimeSpan.",[109,588,589],{"class":553},"FromSeconds",[109,591,592],{"class":119},"(",[109,594,595],{"class":132},"25",[109,597,598],{"class":119},");\n",[109,600,601],{"class":111,"line":144},[109,602,603],{"class":119},"});\n",[109,605,606],{"class":111,"line":156},[109,607,609],{"emptyLinePlaceholder":608},true,"\n",[109,611,612],{"class":111,"line":164},[109,613,614],{"class":175},"\u002F\u002F 映像檔沒有 sleep 可用時，在應用程式裡延後關閉，效果等同 preStop\n",[109,616,617,620,623,626,629,632],{"class":111,"line":179},[109,618,619],{"class":569},"var",[109,621,622],{"class":553}," app",[109,624,625],{"class":569}," =",[109,627,628],{"class":119}," builder.",[109,630,631],{"class":553},"Build",[109,633,634],{"class":119},"();\n",[109,636,637,640,643,646,649,652,655,658,660,662,664],{"class":111,"line":193},[109,638,639],{"class":119},"app.Lifetime.ApplicationStopping.",[109,641,642],{"class":553},"Register",[109,644,645],{"class":119},"(() ",[109,647,648],{"class":569},"=>",[109,650,651],{"class":119}," Thread.",[109,653,654],{"class":553},"Sleep",[109,656,657],{"class":119},"(TimeSpan.",[109,659,589],{"class":553},[109,661,592],{"class":119},[109,663,201],{"class":132},[109,665,666],{"class":119},")));\n",[10,668,669,670,673,674,681],{},"用 ",[70,671,672],{},"ApplicationStopping"," 延後時，這 10 秒是在 SIGTERM 之後才開始算，所以會",[20,675,676,677,680],{},"佔用 ",[70,678,679],{},"ShutdownTimeout"," 的時間","，設定時要一起算進去。",[10,683,684,685,688,689,695],{},"背景工作（例如消費佇列的 ",[70,686,687],{},"BackgroundService","）要",[20,690,691,692],{},"檢查 ",[70,693,694],{},"stoppingToken","，收到取消就停止拿新的工作、把手上這筆做完或放回佇列，而不是做到一半被砍掉。",[227,697,699],{"id":698},"長連線websocket-與-signalr","長連線：WebSocket 與 SignalR",[10,701,702,703,706],{},"HTTP 請求幾秒就結束了，但 WebSocket 連線可能維持好幾個小時，",[20,704,705],{},"舊 Pod 關閉時，連在上面的連線一定會斷","。這沒辦法在伺服器端完全避免，要從架構上處理：",[63,708,709,719,729],{},[30,710,711,714,715,718],{},[20,712,713],{},"用戶端一定要自動重連","：SignalR 用戶端加上 ",[70,716,717],{},"withAutomaticReconnect()","，斷線後會重新連到新的 Pod",[30,720,721,724,725],{},[20,722,723],{},"狀態不能只存在連線上","：例如對話的進度、訂閱的頻道，重連後要能恢復。多實例下的做法可以參考 ",[14,726,728],{"href":727},"\u002Farticles\u002Fsignalr-streaming-gpt","SignalR 串流與多實例部署",[30,730,731,734],{},[20,732,733],{},"滾動更新的速度放慢","：每次只換一個 Pod，避免所有用戶同時重連，把新 Pod 打爆",[49,736,738],{"id":737},"其他中斷poddisruptionbudget","其他中斷：PodDisruptionBudget",[10,740,741,742,745,746,749],{},"除了部署，Pod 也會因為",[20,743,744],{},"計畫中的維護","被趕走：Node 要升級、叢集要縮減 Node 時，管理員會 ",[70,747,748],{},"kubectl drain","，把那台 Node 上的 Pod 全部驅逐。",[10,751,752,753,756,757,760],{},"如果 3 個 Pod 剛好都在同一台 Node 上，drain 時",[20,754,755],{},"3 個會一起被驅逐","，服務就中斷了。",[20,758,759],{},"PodDisruptionBudget（PDB）"," 就是用來限制這件事：",[100,762,764],{"className":102,"code":763,"language":104,"meta":105,"style":105},"apiVersion: policy\u002Fv1\nkind: PodDisruptionBudget\nmetadata:\n  name: house-api\nspec:\n  minAvailable: 2          # 任何時候至少要有 2 個可用\n  selector:\n    matchLabels:\n      app: house-api\n",[70,765,766,776,786,793,802,808,821,828,835],{"__ignoreMap":105},[109,767,768,771,773],{"class":111,"line":112},[109,769,770],{"class":115},"apiVersion",[109,772,129],{"class":119},[109,774,775],{"class":152},"policy\u002Fv1\n",[109,777,778,781,783],{"class":111,"line":123},[109,779,780],{"class":115},"kind",[109,782,129],{"class":119},[109,784,785],{"class":152},"PodDisruptionBudget\n",[109,787,788,791],{"class":111,"line":136},[109,789,790],{"class":115},"metadata",[109,792,120],{"class":119},[109,794,795,798,800],{"class":111,"line":144},[109,796,797],{"class":115},"  name",[109,799,129],{"class":119},[109,801,429],{"class":152},[109,803,804,806],{"class":111,"line":156},[109,805,116],{"class":115},[109,807,120],{"class":119},[109,809,810,813,815,818],{"class":111,"line":164},[109,811,812],{"class":115},"  minAvailable",[109,814,129],{"class":119},[109,816,817],{"class":132},"2",[109,819,820],{"class":175},"          # 任何時候至少要有 2 個可用\n",[109,822,823,826],{"class":111,"line":179},[109,824,825],{"class":115},"  selector",[109,827,120],{"class":119},[109,829,830,833],{"class":111,"line":193},[109,831,832],{"class":115},"    matchLabels",[109,834,120],{"class":119},[109,836,838,841,843],{"class":111,"line":837},9,[109,839,840],{"class":115},"      app",[109,842,129],{"class":119},[109,844,429],{"class":152},[10,846,847],{},"有了 PDB，drain 一次只能驅逐到「剩下 2 個可用」為止，要等被驅逐的 Pod 在其他 Node 重建就緒後，才能繼續驅逐下一個。",[10,849,850],{},"幾個要注意的地方：",[63,852,853,859,868,878],{},[30,854,855,858],{},[20,856,857],{},"PDB 只管「自願中斷」","：drain、叢集自動縮減這類主動驅逐。Node 突然當機、Pod 被 OOMKilled 不在它的控制範圍內",[30,860,861,864,865,867],{},[20,862,863],{},"PDB 不影響 Deployment 的滾動更新","，那是由 ",[70,866,84],{}," 控制的",[30,869,870,873,874,877],{},[20,871,872],{},"不要設得太嚴格","：3 個副本設 ",[70,875,876],{},"minAvailable: 3","，等於任何 Pod 都不能被驅逐，drain 會一直卡住，維運人員只能強制處理",[30,879,880,883,884,887],{},[20,881,882],{},"搭配分散排程","：用 ",[70,885,886],{},"topologySpreadConstraints"," 或 Pod Anti-Affinity，讓副本分散在不同的 Node，一台 Node 出事不會帶走全部",[49,889,890],{"id":890},"新舊版本會同時存在",[10,892,893,894,897,898,241],{},"滾動更新還有一個",[20,895,896],{},"應用程式層面","的影響：更新期間，v1 和 v2 會",[20,899,900],{},"同時在處理請求",[63,902,903,909,915],{},[30,904,905,908],{},[20,906,907],{},"資料庫結構變更要向下相容","：v2 要刪除一個欄位，如果 migration 先跑了，還在運作的 v1 就會出錯。正確做法是分階段：先部署「不再使用這個欄位」的版本，下一次部署才真正刪除欄位（Expand／Contract）",[30,910,911,914],{},[20,912,913],{},"API 的回應格式要相容","：前端可能先拿到 v2 的頁面、再呼叫到 v1 的 API，或反過來",[30,916,917,920],{},[20,918,919],{},"訊息格式要相容","：v2 發出的新格式訊息，可能被還沒更新的 v1 Consumer 收到",[49,922,924],{"id":923},"使用情境物件查詢-api-的部署設定","使用情境：物件查詢 API 的部署設定",[10,926,927],{},"把上面的設定組合起來：",[63,929,930,942,945,957,967],{},[30,931,932,933,936,937,936,939],{},"3 個副本、",[70,934,935],{},"maxSurge: 1","、",[70,938,209],{},[70,940,941],{},"minReadySeconds: 10",[30,943,944],{},"Readiness 檢查資料庫連線與設定載入；Liveness 只檢查行程是否還活著",[30,946,947,949,950,953,954,956],{},[70,948,363],{}," sleep 10 秒，",[70,951,952],{},"terminationGracePeriodSeconds: 45","，ASP.NET Core 的 ",[70,955,679],{}," 25 秒",[30,958,959,960,963,964,966],{},"PDB ",[70,961,962],{},"minAvailable: 2","，搭配 ",[70,965,886],{}," 讓 3 個 Pod 分散在不同 Node",[30,968,969,970,973,974,977],{},"部署時用 ",[70,971,972],{},"kubectl rollout status"," 等待完成，失敗就 ",[70,975,976],{},"kubectl rollout undo"," 退回",[100,979,983],{"className":980,"code":981,"language":982,"meta":105,"style":105},"language-bash shiki shiki-themes github-dark","kubectl rollout status deployment\u002Fhouse-api -n realty --timeout=5m\nkubectl rollout undo deployment\u002Fhouse-api -n realty\n","bash",[70,984,985,1008],{"__ignoreMap":105},[109,986,987,990,993,996,999,1002,1005],{"class":111,"line":112},[109,988,989],{"class":553},"kubectl",[109,991,992],{"class":152}," rollout",[109,994,995],{"class":152}," status",[109,997,998],{"class":152}," deployment\u002Fhouse-api",[109,1000,1001],{"class":132}," -n",[109,1003,1004],{"class":152}," realty",[109,1006,1007],{"class":132}," --timeout=5m\n",[109,1009,1010,1012,1014,1017,1019,1021],{"class":111,"line":123},[109,1011,989],{"class":553},[109,1013,992],{"class":152},[109,1015,1016],{"class":152}," undo",[109,1018,998],{"class":152},[109,1020,1001],{"class":132},[109,1022,1023],{"class":152}," realty\n",[10,1025,1026],{},"部署期間持續對服務打請求驗證，整個滾動更新過程不會出現 5xx。",[49,1028,1029],{"id":1029},"注意事項",[63,1031,1032,1038,1044,1056,1067],{},[30,1033,1034,1037],{},[20,1035,1036],{},"Liveness 不要檢查外部依賴","：資料庫一慢，所有 Pod 的 Liveness 一起失敗、一起被重啟，小問題直接變成全面中斷",[30,1039,1040,1043],{},[20,1041,1042],{},"啟動很慢的應用程式要設 Startup Probe","：否則 Liveness 可能在應用程式還沒啟動完時就判定失敗，Pod 一直重啟",[30,1045,1046,1051,1052,1055],{},[20,1047,1048],{},[70,1049,1050],{},"progressDeadlineSeconds","：新 Pod 一直無法就緒時，Deployment 在這個時間（預設 600 秒）後會標記為失敗。CI\u002FCD 要檢查這個狀態，而不是 ",[70,1053,1054],{},"kubectl apply"," 成功就以為部署完成",[30,1057,1058,538,1061,1063,1064,1066],{},[20,1059,1060],{},"資源要預留",[70,1062,72],{}," 需要額外的資源，叢集剛好用滿時，新 Pod 會卡在 ",[70,1065,217],{},"，部署就卡住了",[30,1068,1069,1072],{},[20,1070,1071],{},"不要只在測試環境驗證","：測試環境流量小，轉發規則的時間差不容易出現問題。正式環境部署時要搭配監控，觀察 5xx 是否在部署期間上升",[49,1074,1075],{"id":1075},"一句話總結",[10,1077,1078,1079,1081],{},"滾動更新要真的不停機，新 Pod 端靠 ",[70,1080,209],{}," 加 Readiness Probe 確保就緒才接流量，舊 Pod 端靠 preStop 等轉發規則更新完、再讓應用程式在寬限時間內處理完手上的請求，長連線要靠用戶端自動重連，Node 維護時靠 PodDisruptionBudget 和分散排程避免一次被趕光；最後別忘了，更新期間新舊版本會同時存在，資料庫與 API 的變更都要向下相容。",[1083,1084,1085],"style",{},"html pre.shiki code .s4JwU, html code.shiki .s4JwU{--shiki-default:#85E89D}html pre.shiki code .s95oV, html code.shiki .s95oV{--shiki-default:#E1E4E8}html pre.shiki code .sDLfK, html code.shiki .sDLfK{--shiki-default:#79B8FF}html pre.shiki code .sU2Wk, html code.shiki .sU2Wk{--shiki-default:#9ECBFF}html pre.shiki code .sAwPA, html code.shiki .sAwPA{--shiki-default:#6A737D}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html pre.shiki code .svObZ, html code.shiki .svObZ{--shiki-default:#B392F0}html pre.shiki code .snl16, html code.shiki .snl16{--shiki-default:#F97583}",{"title":105,"searchDepth":123,"depth":123,"links":1087},[1088,1091,1096,1097,1098,1099,1100],{"id":51,"depth":123,"text":51,"children":1089},[1090],{"id":229,"depth":136,"text":230},{"id":323,"depth":123,"text":324,"children":1092},[1093,1094,1095],{"id":377,"depth":136,"text":378},{"id":502,"depth":136,"text":502},{"id":698,"depth":136,"text":699},{"id":737,"depth":123,"text":738},{"id":890,"depth":123,"text":890},{"id":923,"depth":123,"text":924},{"id":1029,"depth":123,"text":1029},{"id":1075,"depth":123,"text":1075},"2024-04-09","Pod、Deployment、Service 的基本關係可以先參考 Kubernetes 入門。這篇要回答的問題是：Deployment 明明就是滾動更新，為什麼每次部署還是有一小段錯誤？",null,"md",{},"\u002Farticles\u002Fk8s-rolling-update-zero-downtime",{"title":5,"description":1102},"articles\u002Fk8s-rolling-update-zero-downtime",[1110,1111,1112,1113],"Kubernetes","部署","高可用","ASP.NET Core","oSfE-kR88Uf4lImEzvYozRb_KrPw5_gOIxZLbD20dHk",1792959150391]