Database Sharding — ma’lumotni bo‘lib masshtablash

Bitta PostgreSQL serveringiz bor. Foydalanuvchilar o‘sdi, jadval 500 million qatorga yetdi, sekundiga yozuvlar soni ortdi.

Sharding’ning yuragi — routing: kelgan so‘rov qaysi shardga borishini aniqlash. Buni shard key belgilaydi.

Eng muhim farqni yana ta’kidlaymiz: replikatsiyada har node butun ma’lumotni saqlaydi (nusxa), shardingda har node faqat o‘z bo‘lagini. Shuning uchun sharding yozuv va hajmni haqiqatan masshtablaydi.

Ma’lumotni shardlarga taqsimlashning 3 asosiy usuli bor — har biri turli kelishuv bilan:

1 · Range-based (oraliq bo‘yicha)

Shard key qiymat oralig‘i bo‘yicha: ID 1–1M → Shard A, 1M–2M → Shard B. + Oraliq so‘rovlar oson (ketma-ket qatorlar bitta shardda). − Notekislik xavfi: yangi ID’lar doim oxirgi shardga tushib, uni hotspot qiladi.

2 · Hash-based (xesh bo‘yicha) — eng keng tarqalgan

hash(user_id) % N. + Eng tekis taqsimot. − Oraliq so‘rovlar qiyin (qo‘shni qatorlar har xil shardda); va oddiy % N bilan shard qo‘shish/o‘chirish og‘riqli (deyarli hammasi qayta ko‘chadi — buni consistent hashing hal qiladi, Deep’da).

3 · Directory/Lookup (kataloq bo‘yicha)

Alohida lookup jadval qaysi kalit qaysi shardda ekanini saqlaydi. + Eng moslashuvchan (istalgancha ko‘chirish mumkin). − Qo‘shimcha "hop" va lookup jadval o‘zi zaiflik nuqtasi/bottleneck bo‘lishi mumkin.

Yaxshi shard key 3 ta shartni bajaradi

• Yuqori kardinallik — ko‘p turli qiymat (user_id yaxshi; "mamlakat" yomon — atigi bir nechta qiymat).
• Tekis taqsimot — qiymatlar shardlarga teng tarqalsin (hotspot bo‘lmasin).
• So‘rov naqshiga mos — eng ko‘p so‘rovlar bitta shardga tushsin (cross-shard bo‘lmasin).

NestJS’da sharding uchun maxsus paket shart emas — har shardga alohida PrismaClient ulanib, kichik router servisi kalitni to‘g‘ri shardga yo‘naltiradi.

1 · Har shard uchun ulanish

// shard.config.ts — HAR shard uchun alohida Prisma client
const SHARD_URLS = [
  process.env.SHARD_0_URL,   // postgres://...@shard0:5432/shop
  process.env.SHARD_1_URL,   // postgres://...@shard1:5432/shop
  process.env.SHARD_2_URL,
];

2 · Shard router — kalitdan shardni topish

// shard-router.service.ts — kalitni to'g'ri shardga yo'naltiradi
@Injectable()
export class ShardRouter implements OnModuleInit {
  private clients: PrismaClient[] = [];

  onModuleInit() {
    this.clients = SHARD_URLS.map(
      (url) => new PrismaClient({ datasources: { db: { url } } }),
    );
  }

  // shard key (userId) -> qaysi shard (barqaror hash + modulo)
  forUser(userId: string): PrismaClient {
    const hash = createHash('md5').update(userId).digest().readUInt32BE(0);
    return this.clients[hash % this.clients.length];
  }

  all(): PrismaClient[] { return this.clients; }   // cross-shard uchun
}

3 · Foydalanish — so‘rov to‘g‘ri shardga

// orders.service.ts — foydalanuvchining buyurtmalari BITTA shardda (key=userId)
@Injectable()
export class OrdersService {
  constructor(private shards: ShardRouter) {}

  async getUserOrders(userId: string) {
    const db = this.shards.forUser(userId);          // to'g'ri shardni tanla
    return db.order.findMany({ where: { userId } }); // faqat SHU shardga so'rov (tez)
  }

  async create(userId: string, dto: CreateOrderDto) {
    const db = this.shards.forUser(userId);
    return db.order.create({ data: { userId, ...dto } });
  }
}

1 · Cross-shard so‘rov va JOIN

Bir foydalanuvchining ma’lumoti bitta shardda bo‘lsa — tez. Lekin "barcha buyurtmalarni sana bo‘yicha sarala" kabi so‘rov hamma shardga borib, natijani birlashtirishni talab qiladi (scatter-gather) — sekin. Cross-shard JOIN esa umuman ishlamaydi:

// CROSS-SHARD so'rov: hamma shardga borib, natijani birlashtirish ("scatter-gather")
async countAllOrders(): Promise<number> {
  const counts = await Promise.all(
    this.shards.all().map((db) => db.order.count()),   // HAR shardga
  );
  return counts.reduce((a, b) => a + b, 0);            // app tomonida yig'amiz
}
// Sekin va og'ir: imkon qadar QOCHISH kerak (yoki alohida analitika DB ishlatish).

2 · Cross-shard tranzaksiya

Ikki foydalanuvchi turli shardda bo‘lsa, ular orasidagi pul o‘tkazmasi bitta ACID tranzaksiyada bo‘lmaydi (baza chegarasidan oshadi). Yechim — Saga + kompensatsiya (13-mavzu).

3 · Resharding (qayta taqsimlash) — eng og‘riqli

% N ishlatsang, shard qo‘shganda N o‘zgaradi va deyarli barcha ma’lumot ko‘chishi kerak — tizimni to‘xtatmasdan buni qilish juda qiyin. (Yechim: consistent hashing / virtual buckets — Deep qatlamda.)

4 · Hotspot va unique constraint

Yomon key bitta shardni cho‘ktiradi (hotspot). Bundan tashqari, global unique (masalan, email noyobligi) ham qiyinlashadi — chunki tekshiruv hamma shardga borishi kerak.

Sharding’dan OLDIN sinab ko‘ring (tartib bilan)

• 1. Indekslash va so‘rov optimizatsiyasi — ko‘pincha "sekin baza" aslida yomon indeks/so‘rov.
• 2. Keshlash (4-mavzu) — o‘qish yukini Redis bilan keskin kamaytirish.
• 3. O‘qish replikalari (7-mavzu) — o‘qishni replikalarga taqsimlash.
• 4. Vertikal masshtab — kuchliroq server (oddiy va arzonroq, ma’lum chegaragacha).
• 5. Faqat shulardan keyin — sharding.

• Sharding = ma’lumotni bir nechta serverga bo‘lib tashlash (har shard — bo‘lak, nusxa emas). Yozuv va hajmni masshtablaydi. Replikatsiya (nusxa) va partitioning (bitta baza ichida)dan farqli.
• Strategiyalar: range (oraliq, hotspot xavfi), hash (tekis, oraliq qiyin), directory (moslashuvchan, +hop).
• Shard key — eng muhim qaror: yuqori kardinallik + tekis taqsimot + so‘rov naqshiga mos. Timestamp/ketma-ket ID — hotspot.
• Sharding olib keladigan qiyinchiliklar: cross-shard so‘rov/JOIN/tranzaksiya, resharding, hotspot, global unique. ID — UUID/Snowflake.
• Sharding — oxirgi chora. Avval indeks, kesh, o‘qish replikasi, vertikal masshtab. Faqat chinakam yetmaganda shardla.