yceffort
01 POSTS

Web-scraping 1

  • ◆ OG Scraping Server Design Notes Part 1
    #web-scraping#backend#nodejs

    Building an OG Scraping Server in Node.js (1): From Runtime Choice to Error Rate and Latency

    The "10% error rate" of a link preview server mixes five kinds of failure with different causes. This post works out the four points where runtime choice actually matters, lowers the error rate by fixing User-Agent and encoding handling, and verifies the "P95 under one second" target by working backwards from the cache hit rate with a two-million-run simulation. Along the way it shows that Node built-in TextDecoder turns CP949 extension characters into different characters without raising an error.

    33 min read