Node.js stream benchmark: promise, callback va stream

Assalamu Alaykum o'tgan maqolada streamlar haqida to'liq gaplashdik va ularni tezlik va xotirada foydasi tegishini bildirib o'tdik bugun bir muammo olib stream haqiqatdan foydalimi yoqmi buni shu muammo orqali tekshirib ko'ramiz(benchmarking orqali ).

Bu maqolani boshlashdan oldin oldingi maqolani streamlarni yaxshiroq tushunish uchun tavsiya qilaman.link

Muammo uchun o'sha mashhur hamda nodejs dokumentatsiyasida berilgan 1 dan milliongacha bo'lgan sonlarni faylga yozish va o'qishni tanlab oldim .

Unda qani boshladik ;)

Start

Benchmarking Natijalari komputeringizda boshqacha chiqishi mumkin (cpu va RAMga bog'liq).

Birinchi bo'lib biz o'zimiz Nodejsda streamlarni bilmagan holda bu misolga qanday yondashamiz ? Miyamga kelgan birinchi fikr bu fs modul orqali faylga yozish .Birinchi bo'lib shundan boshlaymiz. Bu holda bizda 3 xil usul bor callback funksiyalardan foydalanish ,sinxron funksiyalardan foydalanish yoki promise funksiyalaridan foydalanish .

Promise funksiyalari orqali tekshirib ko'ramiz

unda birinchi bo'lib kodni yozib olamiz .

const fs = require("node:fs/promises");
// Execution Time: 9.3s
// Memory Usage: 50-60MB
(async () => {
  console.time("fs promise solution");
  try {
    const fileHandler = await fs.open("test.txt", "w");
    for (let i = 0; i < 1000000; i++) {
      await fileHandler.write(` ${i} `);
    }
  } catch (error) {
    console.error("An error occurred:", error);
  } finally {
    console.timeEnd("fs promise solution");
  }
})();

(Agarda IIFE haqida kop'roq bilmoqchi bo'lsangiz link ).

memory usage
memory usage

ko'rib turganingizdek yuqoridagi ma'lumotni faylga yozishda promise solution uchun.

Execution Time: 9.3s.

Memory Usage: 50–60MB sarflandi

Callback funksiyalari orqali tekshirib ko'ramiz

const fs = require("node:fs");
// Execution Time: 1.4s
// Memory Usage: 1GB
(async () => {
  console.time("fs callback solution");
  fs.open("test.txt", "w", (err, fd) => {
    if (err) {
      console.error("Error opening file:", err);
      return;
    }
    for (let i = 0; i < 1000000; i++) {
      fs.write(fd, ` ${i} `, (err) => {
        if (err) {
          console.error("Error writing to file:", err);
        }
      });
    }
    console.timeEnd("fs callback solution");
  });
})();

callback versiyadagi yechimda 1.4 soniya va 1GB atrofida xotira sarflandi .

Execution Time: 1.4s.

Memory Usage: 1GB sarflandi

Agarda biz 1 million emas 10 million raqamni yozsak u holat xotira yetishmasligi tufayli pastdagi error kelib chiqadi.

Ko'rib turganingizdek callback holatida xotirada joy yetishmasligi tufayli process o'chirib yuborildi.

Sinxron funksiyalari orqali tekshirib ko'ramiz

const fs = require("node:fs");
// Execution Time: 1.8s
// Memory Usage: 40MB
(async () => {
  console.time("fs sync solution");
  fs.open("test.txt", "w", (err, fd) => {
    if (err) {
      console.error("Error opening file:", err);
      return;
    }
    for (let i = 0; i < 1000000; i++) {
      const buff = Buffer.from(` ${i} `, "utf-8");
      try {
        fs.writeSync(fd, buff);
      } catch (writeErr) {
        console.error("Error writing to file:", writeErr);
        return;
      }
    }
    console.timeEnd("fs sync solution");
  });
})();

Sinxron yechimda esa :

Execution Time: 1.8s.

Memory Usage: 40MB sarflandi.

Tepada 3 ta yechim ko'rdik bularning hammasini har xil yaxshi va yomon tomonlari bor .Agarda yoziladigan sonlarni oshirsak nima bo'ladi ?

  1. Promise solution xotira deyarli o'zgarmay turadi ishlaydigan vaqt ko'payadi .

  2. Callback solution xotira ko'payib xotira yetmasligi natijasida process to'xtatiladi.

  3. Sinxron xotira o'zgarmasdan qoladi ammo vaqt ko'payadi (bu yerda faqat raqam yozilmoqda agar ko'proq vaqt talab qilinadigan task orqali sinxron yozilsa promise solutiondan ko'p vaqt ketadi).

Nodejs dokumentatsiya bo'yicha callback eng tez ishlaydigan usul va bu benchmark orqali ko'rindi ammo bir necha callbacklar xotiraga yozilishi tufayli xotiradan ancha joy olinadi .Ammo tepadagi benchmarkingdan nega sinxron usul promise yechimdan tezroq ishlayapti degan savol keladi chunki raqam yozish ko'p vaqtni olmaydigan bo'lganligi uchun sinxron promise ochib resolve qilish vaqti ketmaganligi evaziga vaqtdan yutmoqda.

Stream solutions

Endi oldingi maqolada aytganimizdek bu turdagi muammolar uchun eng to'g'ri yechim bu streamlar orqali .Biz solishtirish uchun raqamlarni bilib oldik endi stream orqali yechim yozishga o'tamiz.

const fs = require("node:fs/promises");
(async () => {
  try {
    console.time("naive stream solution");
    const fileHandle = await fs.open("test.txt", "w");
    const stream = fileHandle.createWriteStream();
    for (let i = 0; i < 1000000; i++) {
      const buff = Buffer.from(` ${i} `, "utf-8");
      stream.write(buff);
    }
    stream.end(); // Close the stream
    await new Promise((resolve, reject) => {
      stream.on("finish", resolve);
      stream.on("error", reject);
    });
    console.timeEnd("naive stream solution");
  } catch (error) {
    console.error("An error occurred:", error);
  }
})();

Execution Time: 220ms

Memory Usage: 240MB

Bu yechimda ko'rib turganingizdek vaqt ancha tezlashdi 6 marta atrofida ammo biroz ko'proq xotira evaziga .Bu oxirgi yechimmi ?

Yo'q bu oxirgi yechim emas bu yerda biz streamlarda bir narsaga amal qilmayapmiz yani buffer faqat to'lganidan keyingina uni faylga yozish .Bu yechimda har bitta raqam uchun har doim yozilmoqda va bu xotiradan joy olmoqda.

Note: Production da tepadagi yechim tavsiya etilmaydi

Yaxshiroq stream yechimi.

const fs = require("node:fs/promises");
// Execution Time: 245ms
// Memory Usage: 30MB
(async () => {
  console.time("better stream solution");
  try {
    const fileHandle = await fs.open("test.txt", "w");
    const stream = fileHandle.createWriteStream();
    let i = 0;
    const numberOfWrites = 1000000;
    const writeMany = () => {
      while (i < numberOfWrites) {
        const buff = Buffer.from(` ${i} `, "utf-8");
        if (i === numberOfWrites - 1) {
          return stream.end(buff);
        }
        i++;
        if (!stream.write(buff)) break;
      }
    };
    writeMany();
    stream.on("drain", () => {
      writeMany();
    });
    stream.on("finish", () => {
      fileHandle.close();
      console.timeEnd("better stream solution");
    });
  } catch (error) {
    console.error("An error occurred:", error);
  }
})();

Execution Time: 245ms

Memory Usage: 30MB

Kurib turganingizdek shunchki streamdan foydalanish emas streamlardan to'gri foydalanish bizga performance yaxshigina ko'tarishga yordam beradi.

Agarda maqola yoqqan bo'lsa chapak chaling (ko'p chalsayam bo'ladi :).

Xato va kamchiliklar uchun uzr !!!

Kodlarni ushbu githab repoda topishingiz mumkin =>Repo

linkedin.com =>Ulug'bek Habibov | LinkedIn

telegram channel =>@habibov_ulugbek