ํ”ผ๋“œ๋กœ ๋Œ์•„๊ฐ€๊ธฐ
How I Debugged and Fixed Memory & Goroutine Leaks in ProjectDiscovery Nuclei Engine ๐Ÿš€
Dev.toDev.to
Security

LRU ์บ์‹œ ๋ฐ Lifecycle ์ •๋ฆฌ๋ฅผ ํ†ตํ•œ Nuclei ์—”์ง„ ๋ฉ”๋ชจ๋ฆฌ ๋ˆ„์ˆ˜ ์™„๋ฒฝ ํ•ด๊ฒฐ

How I Debugged and Fixed Memory & Goroutine Leaks in ProjectDiscovery Nuclei Engine ๐Ÿš€

ThryLox2026๋…„ 6์›” 28์ผ5๋ถ„intermediate

Context

Nuclei ์—”์ง„์„ ์žฅ๊ธฐ ์‹คํ–‰ ๋งˆ์ดํฌ๋กœ์„œ๋น„์Šค์— SDK ํ˜•ํƒœ๋กœ ์ž„๋ฒ ๋”ฉ ์‹œ ๋ฉ”๋ชจ๋ฆฌ ํŒฝ์ฐฝ ๋ฐ Goroutine ๋ˆ„์ˆ˜ ๋ฐœ์ƒ. ํŠนํžˆ unbounded sync.Map ์‚ฌ์šฉ๊ณผ ๋ช…์‹œ์  ์ž์› ํ•ด์ œ ์ ˆ์ฐจ ๋ถ€์žฌ๋กœ ์ธํ•œ ์‹œ์Šคํ…œ ๋ถˆ์•ˆ์ •์„ฑ ํ™•์ธ.

Technical Solution

  • unbounded sync.Map์„ 4,096 entries ์ œํ•œ ๋ฐ 24์‹œ๊ฐ„ TTL ๊ธฐ๋ฐ˜์˜ expirable LRU Cache๋กœ ๊ต์ฒดํ•˜์—ฌ ๋ฉ”๋ชจ๋ฆฌ ์ƒํ•œ์„  ์„ค์ •
  • protocolstate.Close() ๋‚ด PerHostRateLimitPool ํ•ด์ œ ๋กœ์ง์„ ์ถ”๊ฐ€ํ•˜์—ฌ ๊ณ ๋ฆฝ๋œ Goroutine ์ œ๊ฑฐ
  • NucleiEngine.Close() ์‹œ์ ์— Template Parser์˜ Purge() ๋ฉ”์„œ๋“œ๋ฅผ ํ˜ธ์ถœํ•˜์—ฌ AST ์บ์‹œ ๊ฐ•์ œ ์ œ๊ฑฐ
  • Interface Type Assertion์„ ํ™œ์šฉํ•ด ์ปค์Šคํ…€ ํŒŒ์„œ ์‚ฌ์šฉ ์‹œ์—๋„ ๊ฒฐํ•ฉ๋„๋ฅผ ๋‚ฎ์ถ”๋ฉฐ ์ž์› ์ •๋ฆฌ ๊ธฐ๋Šฅ์„ ์ˆ˜ํ–‰ํ•˜๋Š” ์œ ์—ฐํ•œ ๊ตฌ์กฐ ์„ค๊ณ„
  • Engine Lifecycle ๋‹จ๊ณ„์— ๋งž์ถ˜ ์ˆœ์ฐจ์  Teardown ํ”„๋กœ์„ธ์Šค ๊ตฌ์ถ•์œผ๋กœ ๋ฆฌ์†Œ์Šค ๋ˆ„์ˆ˜ ๊ฒฝ๋กœ ์ฐจ๋‹จ

- long-running ์•ฑ์—์„œ sync.Map ์‚ฌ์šฉ ์‹œ TTL ๋ฐ ์ตœ๋Œ€ ํฌ๊ธฐ ์ œํ•œ์ด ์žˆ๋Š” LRU ์บ์‹œ ๊ฒ€ํ†  - SDK ์„ค๊ณ„ ์‹œ Close() ๋˜๋Š” Purge() ๋ฉ”์„œ๋“œ๋ฅผ ํ†ตํ•œ ๋ฐฑ๊ทธ๋ผ์šด๋“œ Goroutine ๋ฐ ์ฑ„๋„์˜ ๋ช…์‹œ์  ์ข…๋ฃŒ ๊ตฌํ˜„ - ๊ฐ•๋ ฅํ•œ ํƒ€์ž… ์˜์กด์„ฑ ์—†์ด ๊ธฐ๋Šฅ์„ ํ™•์žฅํ•˜๊ธฐ ์œ„ํ•ด Interface Type Assertion ๊ธฐ๋ฐ˜์˜ Lifecycle Hook ์ ์šฉ

์›๋ฌธ ์ฝ๊ธฐ