Rânduri duplicate de date de facturare în cloud Google

Am extras datele de facturare GCP prin BigQuery. Am făcut interogări pentru a prelua date pentru o anumită durată de timp și apoi am găsit două tipuri de date duplicat:

1. datele duplicate au rezultat din anularea coloanei de etichetă

2. date duplicate rezultate din rânduri de interogare normală (ca mai jos) cu aceeași valoare pentru toate coloanele.

      billing_account_id AS id,
      service.id AS ServiceId,
      service.description AS ServiceDescription,
      sku.id AS SkuId,
      sku.description AS SkuDescription,
      usage_start_time,
      usage_end_time,
      project.id AS ProjectId,
      project.number AS ProjectNumber,
      project.name AS ProjectName,
      project.labels ca ProjectLabels,
      project.ancestry_numbers AS ProjectAncestryNumbers,
      etichete AS etichete,
      system_labels AS SystemLabels,
      locație.locație AS Locație,
      locație.țară AS Țara,
      locație.regiune Regiune AS,
      location.zone AS Zone,
      SUBSTRING(cast(export_time ca șir), 1, 19) AS ExportTime,
      cost AS Cost,
      valuta AS Moneda,
      currency_conversion_rate AS CurrencyConversionRate,
      usage.amount AS UsageAmount,
      usage.unit AS UsageUnit,
      usage.amount_in_pricing_units AS UsageAmountInPricingUnits,
      usage.pricing_unit AS UsagePricingUnit,
      credite AS Credite,
      invoice.month AS InvoiceMonth,
      cost_type AS CostType,
      adjustment_info.id AS adjustmentInfoId,
      adjustment_info.description AS adjustmentInfoDescription,
      adjustment_info.mode AS adjustmentInfoMode,
      adjustment_info.type AS adjustmentInfoType
    DIN 
    „NUMELE TABELUI” 
    WHERE DATE(_PARTITIONTIME) BETWEEN subtractFromTime(utcnow(),60,'Day','yyyy-LL-zz')ȘI formatDateTime(utcnow(),'yyyy-LL-dd')```


**Aș dori să știu de ce există date duplicat? Cum pot fi gestionate și cum pot fi siguri că calculul costurilor nu a fost afectat de datele duplicate?**

Apreciez daca ma poate ajuta cineva.


Cele mai bune,
Shokoufeh

1. Why rows get duplicated when un-nesting the labels field?

When you are un-nesting a repeated type field like labels, duplication of rows is expected. To be precise, each row gets duplicated based on the length of that row's labels array.

2. Why are there duplicate rows even before un-nesting?

If you create 2 Compute Engine VMs of exact config and location, the idle usage rows in the billing export of those 2 VMs are exactly the same unless labelled. The export table doesn't have an explicitly exposed primary key.

The export table's granularity is only until service & SKUs and not per resource. This leads to duplicate looking data but they are actual valid usages.