You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-12-08 - 03:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Thu Dec 07, 2023 12:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
146686422750,9sleep10-21swapper/107:05:271
154672821230,17sleep234-21ksoftirqd/210:50:342
149794021000,5sleep20-21swapper/208:30:442
14680239910086,11cyclictest0-21swapper/108:31:591
146770029066,16sleep20-21swapper/207:07:382
146766829070,14sleep00-21swapper/007:07:120
1468019998679,4cyclictest0-21swapper/008:31:590
146766128656,23sleep30-21swapper/307:07:073
162399830,8rtkit-daemon881251-21Softwar~cThread10:17:182
1468023998173,5cyclictest0-21swapper/111:47:591
1468023998173,5cyclictest0-21swapper/111:47:591
146802599771,53cyclictest1753-21gnome-shell08:18:592
146801999771,71cyclictest880898-21Xwayland08:01:240
162399750,10rtkit-daemon880970-21Timer10:09:052
1468023997517,31cyclictest1582400-21wc12:35:021
162399740,4rtkit-daemon0-21swapper/208:23:332
14912022740,4chrt1491201-21sh08:15:000
1468019997365,5cyclictest0-21swapper/011:47:590
1468019997365,5cyclictest0-21swapper/011:47:590
162399720,7rtkit-daemon961-21snmpd07:24:502
146802399722,46cyclictest881331-21Privileged14:31:441
162399710,8rtkit-daemon961-21snmpd09:48:182
162399710,4rtkit-daemon0-21swapper/207:11:492
1468023996946,7cyclictest26-21ksoftirqd/111:16:241
162399680,9rtkit-daemon1775-21gvfs-afc-volume11:33:562
162399680,3rtkit-daemon0-21swapper/211:03:212
162399670,4rtkit-daemon0-21swapper/209:07:272
15800852670,5sleep10-21swapper/112:25:401
146802599671,13cyclictest881253-21Renderer07:46:242
146802399642,6cyclictest0-21swapper/109:48:581
146801999642,36cyclictest1573669-21systemd12:10:020
162399630,8rtkit-daemon1622-21rtkit-daemon07:34:073
146802399631,29cyclictest0-21swapper/107:46:241
162399620,9rtkit-daemon1622-21rtkit-daemon11:44:052
146802999623,15cyclictest1753-21gnome-shell10:46:243
162399610,9rtkit-daemon1622-21rtkit-daemon11:25:413
146802999613,51cyclictest1753-21gnome-shell12:16:593
146802599611,20cyclictest15133372swap09:15:372
146802399619,47cyclictest26-21ksoftirqd/109:01:231
1468023996145,13cyclictest231irq_work/107:52:591
1468023996145,13cyclictest231irq_work/107:52:591
146801999618,26cyclictest0-21swapper/011:43:370
146801999612,15cyclictest881253-21Renderer10:16:240
162399600,9rtkit-daemon1622-21rtkit-daemon10:21:102
162399600,10rtkit-daemon1622-21rtkit-daemon09:57:343
146802999601,46cyclictest1753-21gnome-shell10:12:593
146801999604,33cyclictest1617-10pipewire-media-07:20:460
146801999602,54cyclictest0-21swapper/009:48:590
1468019996013,41cyclictest881251-21Softwar~cThread07:28:510
146801999601,18cyclictest1753-21gnome-shell09:23:590
162399590,9rtkit-daemon1622-21rtkit-daemon09:36:422
162399590,9rtkit-daemon1622-21rtkit-daemon09:36:422
146802999591,17cyclictest1753-21gnome-shell08:17:593
146801999592,33cyclictest1477301-21munin-run07:35:010
1468019995916,38cyclictest908-21gdbus10:33:520
1468019995915,39cyclictest1546269-21expr10:50:220
162399580,9rtkit-daemon1622-21rtkit-daemon10:29:082
162399580,9rtkit-daemon1622-21rtkit-daemon09:53:092
162399580,9rtkit-daemon1622-21rtkit-daemon09:17:463
162399580,9rtkit-daemon1622-21rtkit-daemon09:02:093
162399580,9rtkit-daemon1622-21rtkit-daemon08:41:312
15339282580,9sleep31533623-21Classif~14:31:443
146802999582,52cyclictest1753-21gnome-shell11:19:583
146802999581,53cyclictest1753-21gnome-shell12:10:593
146802999581,53cyclictest1753-21gnome-shell11:03:593
146802999581,16cyclictest1753-21gnome-shell11:44:583
1468025995815,11cyclictest1753-21gnome-shell11:16:242
1468023995812,40cyclictest1753-21gnome-shell07:18:581
1468019995811,41cyclictest881251-21Softwar~cThread10:48:020
162399570,9rtkit-daemon1622-21rtkit-daemon08:07:492
162399570,8rtkit-daemon1622-21rtkit-daemon07:14:273
1468023995739,15cyclictest0-21swapper/108:18:591
146802399572,51cyclictest1753-21gnome-shell07:10:581
146802399572,11cyclictest881253-21Renderer12:16:241
1468019995726,26cyclictest880898-21Xwayland07:46:240
162399560,9rtkit-daemon1622-21rtkit-daemon10:39:253
162399560,9rtkit-daemon1622-21rtkit-daemon10:31:423
162399560,9rtkit-daemon1622-21rtkit-daemon09:32:052
162399560,9rtkit-daemon1622-21rtkit-daemon08:30:563
162399560,8rtkit-daemon1622-21rtkit-daemon11:09:443
146802599561,51cyclictest1753-21gnome-shell10:59:592
146802599561,51cyclictest1753-21gnome-shell10:59:582
146802399562,12cyclictest1753-21gnome-shell07:57:591
146801999564,46cyclictest0-21swapper/009:01:240
1468019995623,27cyclictest438-20systemd-journal08:39:380
1468019995613,37cyclictest881251-21Softwar~cThread09:26:010
162399550,9rtkit-daemon1622-21rtkit-daemon09:11:073
162399550,9rtkit-daemon1622-21rtkit-daemon09:11:073
162399550,9rtkit-daemon1622-21rtkit-daemon08:57:333
162399550,9rtkit-daemon1622-21rtkit-daemon08:49:063
162399550,9rtkit-daemon1622-21rtkit-daemon08:49:053
162399550,8rtkit-daemon1622-21rtkit-daemon08:11:053
146802999551,49cyclictest1753-21gnome-shell11:33:593
1468025995517,12cyclictest1541289-21wc10:35:302
1468023995539,13cyclictest0-21swapper/110:01:591
146802399552,10cyclictest1753-21gnome-shell07:29:581
146802399551,11cyclictest1753-21gnome-shell07:37:581
1468019995513,37cyclictest881251-21Softwar~cThread11:51:360
1468019995513,37cyclictest881251-21Softwar~cThread09:08:310
162399540,9rtkit-daemon1622-21rtkit-daemon11:09:382
162399540,9rtkit-daemon1622-21rtkit-daemon07:30:552
162399540,8rtkit-daemon1622-21rtkit-daemon08:03:513
162399540,8rtkit-daemon1622-21rtkit-daemon07:50:043
162399540,8rtkit-daemon1622-21rtkit-daemon07:50:043
146802999542,41cyclictest1753-21gnome-shell12:32:583
146802599542,46cyclictest881251-21Softwar~cThread12:05:592
1468025995419,11cyclictest1568530-21find11:55:202
1468025995418,11cyclictest2962348-21rs:main2
1468025995416,12cyclictest1502045-21sh08:45:212
1468025995416,12cyclictest1502045-21sh08:45:212
146802399542,48cyclictest231irq_work/108:52:591
146801999542,48cyclictest1545745-21kworker/0:212:18:590
1468019995420,29cyclictest1558110-21find11:25:190
146802999532,10cyclictest1753-21gnome-shell10:54:583
146802599533,46cyclictest0-21swapper/212:16:592
146802599532,11cyclictest1453513-21kworker/2:107:26:582
146802599532,10cyclictest1753-21gnome-shell11:12:592
1468025995317,11cyclictest1542559-21cat10:40:182
1468025995314,13cyclictest880726-21firefox11:50:022
1468025995314,13cyclictest880726-21firefox11:50:012
1468025995313,12cyclictest881251-21Softwar~cThread12:38:152
146802399531,10cyclictest1753-21gnome-shell07:44:591
1468019995336,10cyclictest1555817-21BgIOThr~0
146801999532,46cyclictest1753-21gnome-shell07:12:580
146801999532,45cyclictest881251-21Softwar~cThread09:11:130
146801999532,45cyclictest881251-21Softwar~cThread09:11:130
1468019995320,27cyclictest438-20systemd-journal09:30:380
1468019995313,34cyclictest881251-21Softwar~cThread08:06:530
162399520,8rtkit-daemon1622-21rtkit-daemon07:23:233
162399520,26rtkit-daemon1622-21rtkit-daemon07:27:473
1468025995237,10cyclictest1487444-21taskset08:01:242
1468025995228,20cyclictest1538984-40lsb_release10:30:192
1468025995219,8cyclictest1581605-21ntpq12:30:332
1468025995215,11cyclictest1562025-21basename11:35:272
1468025995212,13cyclictest881251-21Softwar~cThread12:29:322
1468019995220,27cyclictest1540878-21cat10:35:210
1468019995219,28cyclictest1513493-21cat09:15:390
162399510,6rtkit-daemon1622-21rtkit-daemon08:51:323
1468029995137,9cyclictest1753-21gnome-shell11:47:593
1468029995137,9cyclictest1753-21gnome-shell11:47:593
146802599518,38cyclictest1753-21gnome-shell09:04:592
146802599512,46cyclictest0-21swapper/207:52:592
146802599512,46cyclictest0-21swapper/207:52:592
1468025995118,11cyclictest1479291-21sed07:40:022
1468025995118,10cyclictest1514461-21expr09:20:202
1468025995117,11cyclictest1522012-21ntp_kernel_pll_09:40:322
146802599511,45cyclictest1753-21gnome-shell08:12:592
146802399515,42cyclictest1503485-21cron08:50:001
146802399515,42cyclictest1503485-21cron08:50:001
1468023995139,8cyclictest1753-21gnome-shell08:08:591
1468023995137,10cyclictest231irq_work/110:16:251
146802399512,8cyclictest231irq_work/109:57:591
1468023995117,29cyclictest438-20systemd-journal09:39:551
1468023995117,29cyclictest438-20systemd-journal09:39:551
146801999514,42cyclictest1521705-21ls09:40:270
1468019995143,5cyclictest0-21swapper/008:52:590
146801999512,45cyclictest0-21swapper/009:39:590
146801999512,45cyclictest0-21swapper/009:39:590
146801999512,45cyclictest0-21swapper/008:46:590
146801999512,45cyclictest0-21swapper/008:46:590
146801999512,45cyclictest0-21swapper/008:20:590
1468019995119,27cyclictest1477887-21cat07:35:210
1468019995118,28cyclictest1583490-21awk12:35:390
162399500,6rtkit-daemon1622-21rtkit-daemon07:55:503
146802999502,44cyclictest1753-21gnome-shell10:02:593
146802599504,44cyclictest1468013-21cyclictest12:00:592
1468025995042,5cyclictest0-21swapper/211:54:592
1468025995020,7cyclictest1069-21runrttasks07:18:372
1468023995017,28cyclictest881331-21Privileged14:31:441
146802399501,46cyclictest1501813-21kworker/1:211:39:591
146802399501,28cyclictest0-21swapper/112:07:591
1468023995011,33cyclictest885426-21Web14:31:441
1468019995033,14cyclictest0-21swapper/010:01:590
146801999502,7cyclictest0-21swapper/012:32:590
146801999502,39cyclictest0-21swapper/011:31:590
1468019995019,27cyclictest1495437-21expr08:25:230
1468019995019,26cyclictest1537064-21find10:25:180
1468019995018,27cyclictest1551427-21expr11:05:220
146802999491,44cyclictest1753-21gnome-shell09:38:593
146802999491,44cyclictest1753-21gnome-shell09:38:593
1468025994918,7cyclictest1069-21runrttasks07:42:562
1468023994939,7cyclictest1401250-21kworker/1:008:37:591
146802399493,44cyclictest231irq_work/107:24:581
146802399492,7cyclictest0-21swapper/111:06:581
146802399492,5cyclictest0-21swapper/112:13:591
146802399492,44cyclictest0-21swapper/110:37:591
146802399492,44cyclictest0-21swapper/108:59:591
146802399492,43cyclictest0-21swapper/111:23:591
1468023994924,21cyclictest1517397-21latency_hist09:30:021
1468023994913,31cyclictest1539216-21expr10:30:211
1468023994913,30cyclictest1547726-21cpu10:55:251
1468023994913,30cyclictest1547726-21cpu10:55:251
1468023994912,31cyclictest1513134-21netstat09:15:321
146801999492,6cyclictest0-21swapper/010:44:590
146801999492,6cyclictest0-21swapper/008:41:590
146801999492,44cyclictest0-21swapper/009:57:590
1468019994916,28cyclictest1550496-21cat11:00:400
1468019994916,27cyclictest1574190-21cat12:10:230
146802999482,7cyclictest0-21swapper/312:08:583
146802599486,21cyclictest1528784-40lsb_release10:00:272
146802599482,7cyclictest0-21swapper/208:57:592
146802599482,6cyclictest0-21swapper/212:24:592
146802599482,5cyclictest0-21swapper/208:52:592
146802599482,43cyclictest1485676-21runrttasks07:55:592
146802599482,43cyclictest0-21swapper/211:21:592
146802599482,43cyclictest0-21swapper/210:10:592
146802599482,43cyclictest0-21swapper/208:39:592
146802599482,43cyclictest0-21swapper/208:29:592
1468025994817,7cyclictest1527501-21timerandwakeup09:55:422
1468023994839,6cyclictest231irq_work/109:14:591
1468023994839,6cyclictest231irq_work/109:14:591
146802399482,5cyclictest0-21swapper/111:56:591
146802399482,5cyclictest0-21swapper/110:52:591
146802399482,43cyclictest0-21swapper/111:30:581
146802399482,43cyclictest0-21swapper/111:29:591
146802399482,42cyclictest0-21swapper/109:05:591
146802399481,44cyclictest231irq_work/109:22:591
1468023994813,30cyclictest1476466-21needreboot07:30:311
1468019994840,5cyclictest0-21swapper/011:35:590
1468019994840,5cyclictest0-21swapper/007:41:590
1468019994840,5cyclictest0-21swapper/007:16:580
146801999482,6cyclictest0-21swapper/012:04:580
146801999482,6cyclictest0-21swapper/008:15:590
146801999482,43cyclictest0-21swapper/007:55:590
146801999482,41cyclictest1-21systemd12:25:010
1468019994815,28cyclictest1532234-21cat10:10:210
1468019994814,29cyclictest1536031-21strings10:20:320
1468025994739,5cyclictest0-21swapper/209:12:592
1468025994739,5cyclictest0-21swapper/209:12:592
146802599472,42cyclictest0-21swapper/212:12:582
146802599472,42cyclictest0-21swapper/210:47:592
146802599472,42cyclictest0-21swapper/209:28:592
146802399474,40cyclictest1531604-21grep10:10:001
1468023994739,5cyclictest0-21swapper/109:43:591
1468023994734,8cyclictest881253-21Renderer09:31:241
146802399472,5cyclictest0-21swapper/112:20:581
146802399472,5cyclictest0-21swapper/108:22:591
146802399472,43cyclictest0-21swapper/111:42:591
146802399472,42cyclictest0-21swapper/111:12:591
146802399472,42cyclictest0-21swapper/110:45:591
1468023994713,29cyclictest1549663-21cstates11:00:221
1468019994739,5cyclictest0-21swapper/008:58:590
146801999472,43cyclictest0-21swapper/010:06:590
146801999472,42cyclictest0-21swapper/011:56:590
1468019994718,25cyclictest1482801-21basename07:50:180
1468019994718,25cyclictest1482801-21basename07:50:180
1468029994634,8cyclictest1753-21gnome-shell11:11:593
1468023994639,5cyclictest231irq_work/110:13:591
1468023994639,4cyclictest0-21swapper/110:20:591
1468023994638,5cyclictest0-21swapper/112:00:591
1468023994620,22cyclictest1500277-21/usr/sbin/munin08:40:281
1468023994612,29cyclictest1537328-21cat10:25:211
1468019994638,5cyclictest0-21swapper/009:53:590
1468019994617,24cyclictest1549363-21wc11:00:020
1468019994617,24cyclictest1549363-21wc11:00:020
1468019994613,28cyclictest1557149-21open_files11:20:330
1468029994529,12cyclictest1753-21gnome-shell12:24:593
1468029994524,17cyclictest1571797-21munin-run12:05:013
146802399459,31cyclictest1525014-21cstates09:50:211
1468023994520,21cyclictest1069-21runrttasks08:11:431
1468019994539,4cyclictest0-21swapper/011:12:590
1468029994431,9cyclictest1753-21gnome-shell10:08:593
1468029994428,12cyclictest1753-21gnome-shell10:28:593
1468029994424,16cyclictest42-21ksoftirqd/307:40:233
1468023994435,6cyclictest231irq_work/108:29:591
1468023994417,22cyclictest1567021-21cstates11:50:211
1468019994436,5cyclictest0-21swapper/012:25:590
1468029994317,22cyclictest1517843-40lsb_release09:30:213
1468029994311,28cyclictest855-21dbus-daemon09:46:523
162399420,36rtkit-daemon0-21swapper/309:24:213
146802999427,6cyclictest141rcu_preempt08:40:003
146802999419,13cyclictest881251-21Softwar~cThread12:37:303
1468029994135,4cyclictest0-21swapper/307:48:593
146802999411,11cyclictest438-20systemd-journal08:44:553
1468029994023,13cyclictest1753-21gnome-shell08:25:593
1468029994017,18cyclictest2962348-21rs:main3
162399390,32rtkit-daemon0-21swapper/311:57:293
146802999391,33cyclictest1753-21gnome-shell10:41:593
162399380,30rtkit-daemon881251-21Softwar~cThread08:21:493
1468029993827,7cyclictest0-21swapper/308:07:593
1468029993826,8cyclictest1753-21gnome-shell09:25:593
146802999381,32cyclictest1753-21gnome-shell10:22:593
1468029993720,12cyclictest1556960-21sed11:20:283
146802999371,31cyclictest1753-21gnome-shell12:26:593
1468029993621,11cyclictest1522019-21ntp_kernel_pll_09:40:323
1468029993617,14cyclictest881331-21Privileged14:31:443
1468029993617,14cyclictest881331-21Privileged14:31:443
146802999358,11cyclictest881251-21Softwar~cThread09:52:073
1468029993419,11cyclictest1509520-21cat09:05:303
1468029993418,12cyclictest1470269-21cat07:15:123
1468029993416,13cyclictest881251-21Softwar~cThread11:54:483
1468029993318,11cyclictest1478581-21sed07:35:393
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional