[Busla]

так надо писать на устройство, а не в ФС

[Jula0071]

dd if=/dev/zero of=/mnt/output oflag=dsync conv=fdatasync bs=50M count=100 »

Ну и где результат-то? Вы его забыли показать, что говорит о том, что вы не понимаете, что делаете. Какая фс используется также не сказано, вдруг вы там ntfs через fuse. О том, что dd не тот инструмент, чтоб измерять скорость, умолчим.

[made_in_Chuguev]

Файловая система xfs.
Для сравнения приведу характеристики и тест двух серверов. На одном скорость работы SSD нормальная, а со вторым вот такая фигня.

Сервер 1 (на котором все норм.)

lshw -short -C system -C processor -C bridge
H/W path Device Class Description
===========================================================
system System Product Name (SKU)
/0/4c processor Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
/0/100 bridge Intel Corporation
/0/100/1 bridge Skylake PCIe Controller (x16)
/0/100/1b bridge 200 Series PCH PCI Express Root Port #17
/0/100/1c.2/0 bridge ASM1083/1085 PCIe to PCI Bridge

nvme list
Node SN Model Namespace Usage Format FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1 S3ETNX0J604697N Samsung SSD 960 EVO 1TB 1 637.02 GB / 1.00 TB 512 B + 0 B 2B7QCXE7

lspci -v -s 07:00.0
07:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM961/PM961 (prog-if 02 [NVM Express])
Subsystem: Samsung Electronics Co Ltd Device a801
Flags: bus master, fast devsel, latency 0, IRQ 16, NUMA node 0
Memory at f7000000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable- Count=1/32 Maskable- 64bit+
Capabilities: [70] Express Endpoint, MSI 00
Capabilities: [b0] MSI-X: Enable+ Count=8 Masked-
Capabilities: [100] Advanced Error Reporting
Capabilities: [148] Device Serial Number 00-00-00-00-00-00-00-00
Capabilities: [158] Power Budgeting <?>
Capabilities: [168] #19
Capabilities: [188] Latency Tolerance Reporting
Capabilities: [190] L1 PM Substates
Kernel driver in use: nvme

hdparm -tT --direct /dev/nvme0n1

Timing O_DIRECT cached reads: 2550 MB in 2.00 seconds = 1274.90 MB/sec
Timing O_DIRECT disk reads: 6792 MB in 3.00 seconds = 2263.83 MB/sec


dd if=/dev/zero of=/mnt-samsung-nvme0n1p1/output oflag=dsync conv=fdatasync bs=50M count=100

5242880000 bytes (5.2 GB, 4.9 GiB) copied, 5.17535 s, 1.0 GB/s

Сервер 2 (на котором есть проблема)

lshw -short -C system -C processor -C bridge
H/W path Device Class Description
==========================================================
system PowerEdge R420 (SKU=NotProvided;ModelName=PowerEdge R420)
/0/400 processor Intel(R) Xeon(R) CPU E5-2430L 0 @ 2.00GHz
/0/401 processor Intel(R) Xeon(R) CPU E5-2430L 0 @ 2.00GHz
/0/100 bridge Xeon E5/Core i7 DMI2
/0/100/1 bridge Xeon E5/Core i7 IIO PCI Express Root Port 1a
/0/100/3 bridge Xeon E5/Core i7 IIO PCI Express Root Port 3a in PCI Express Mode
/0/100/11 bridge C600/X79 series chipset PCI Express Virtual Root Port
/0/100/1c bridge C600/X79 series chipset PCI Express Root Port 1
/0/100/1c.4 bridge C600/X79 series chipset PCI Express Root Port 5
/0/100/1c.7 bridge C600/X79 series chipset PCI Express Root Port 8
/0/100/1c.7/0 bridge SH7757 PCIe Switch [PS]
/0/100/1c.7/0/0 bridge SH7757 PCIe Switch [PS]
/0/100/1c.7/0/0/0 bridge SH7757 PCIe-PCI Bridge [PPB]
/0/100/1c.7/0/1 bridge SH7757 PCIe Switch [PS]
/0/100/1e bridge 82801 PCI Bridge
/0/100/1f bridge C600/X79 series chipset LPC Controller
/0/3 bridge Xeon E5/Core i7 IIO PCI Express Root Port 3a in PCI Express Mode

nvme list
Node SN Model Namespace Usage Format FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1 S3X3NF0JA01887K Samsung SSD 960 EVO 1TB 1 488.83 MB / 1.00 TB 512 B + 0 B 3B7QCXE7

lspci -v -s 41:00.0
41:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd Device a804 (prog-if 02 [NVM Express])
Subsystem: Samsung Electronics Co Ltd Device a801
Flags: bus master, fast devsel, latency 0, IRQ 33, NUMA node 1
Memory at d40fc000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable- Count=1/32 Maskable- 64bit+
Capabilities: [70] Express Endpoint, MSI 00
Capabilities: [b0] MSI-X: Enable+ Count=8 Masked-
Capabilities: [100] Advanced Error Reporting
Capabilities: [148] Device Serial Number 00-00-00-00-00-00-00-00
Capabilities: [158] Power Budgeting <?>
Capabilities: [168] #19
Capabilities: [188] Latency Tolerance Reporting
Capabilities: [190] L1 PM Substates
Kernel driver in use: nvme

hdparm -tT --direct /dev/nvme0n1

/dev/nvme0n1:
Timing O_DIRECT cached reads: 3604 MB in 2.00 seconds = 1804.39 MB/sec
Timing O_DIRECT disk reads: 7902 MB in 3.00 seconds = 2633.84 MB/sec


dd if=/dev/zero of=/mnt/output oflag=dsync conv=fdatasync bs=50M count=100

5242880000 bytes (5.2 GB, 4.9 GiB) copied, 9.60919 s, 546 MB/s

[Jula0071]

made_in_Chuguev, покажите, с какими опциями смонтированы диски на обоих серверах.
hdparm показывает более-менее нормальные цифры. dd не тот инструмент, прогоните fio. https://wiki.mikejung.biz/Benchmarking#Fio

По поводу скорости, встроенный драйвер nvme очень простой, но есть асинхронный драйвер SPDK. Правда, его пока не включили в proxmox.