You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-28 - 22:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack8slot6s.osadl.org (updated Fri Aug 28, 2026 12:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
168979910521050,2cyclictest6834-21kworker/0:0+events08:06:490
168979910521050,2cyclictest29387-21kworker/0:0+events09:47:560
168979910521050,2cyclictest19595-21kworker/0:2+events08:48:000
168979910521050,2cyclictest1866-21kworker/0:2+events07:48:220
168979910521049,3cyclictest26229-21kworker/0:0+events10:12:110
169139910511049,2cyclictest12402-21kworker/3:0+events07:21:283
169139910511047,4cyclictest22544-21kworker/3:2+events10:06:043
168979910511049,2cyclictest3914-21kworker/0:1+events12:28:500
168979910511049,2cyclictest3914-21kworker/0:1+events12:24:000
168979910511049,2cyclictest29387-21kworker/0:0+events09:23:040
168979910511049,2cyclictest27531-21kworker/0:1+events10:51:150
168979910511049,2cyclictest25463-21kworker/0:0+events11:57:560
168979910511049,2cyclictest2305-21kworker/0:2+events09:53:250
168979910511049,2cyclictest17555-21kworker/0:0+events08:34:040
168979910511048,3cyclictest15868-21kworker/0:1+events11:48:080
169139910501048,2cyclictest4183-21kworker/3:1+events10:22:083
169139910501048,2cyclictest12402-21kworker/3:0+events08:18:023
169139910501048,2cyclictest12402-21kworker/3:0+events07:31:363
169139910501047,3cyclictest4183-21kworker/3:1+events11:40:453
169139910501047,3cyclictest12402-21kworker/3:0+events07:12:363
169139910501046,2cyclictest24746-21kworker/3:1+events07:28:413
168979910501048,2cyclictest29387-21kworker/0:0+events09:55:230
168979910501048,2cyclictest29387-21kworker/0:0+events08:56:060
168979910501048,2cyclictest27531-21kworker/0:1+events10:55:440
168979910501048,2cyclictest2720-21kworker/0:1+events08:37:350
168979910501048,2cyclictest1866-21kworker/0:2+events07:59:070
168979910501048,2cyclictest11984-21kworker/0:2+events11:12:120
168979910501047,3cyclictest17765-21kworker/0:0+events10:43:190
169139910491045,2cyclictest32697-21kworker/3:1+events07:15:223
168979910491047,2cyclictest4855-21kworker/0:0+events11:03:570
168979910491047,2cyclictest3914-21kworker/0:1+events12:06:070
168979910491047,2cyclictest2720-21kworker/0:1+events08:21:280
168979910491047,2cyclictest2720-21kworker/0:1+events08:13:080
168979910491047,2cyclictest19595-21kworker/0:2+events08:44:440
168979910491047,2cyclictest11984-21kworker/0:2+events12:02:090
168979910491046,3cyclictest11984-21kworker/0:2+events10:45:200
169139910481044,2cyclictest6212-21kworker/3:0+events11:35:423
168979910481046,2cyclictest6574-21kworker/0:1+events10:27:140
168979910481046,2cyclictest2720-21kworker/0:1+events08:50:430
168979910481045,3cyclictest11984-21kworker/0:2+events11:21:340
168979910471045,2cyclictest2720-21kworker/0:1+events07:51:500
168979910461044,2cyclictest11984-21kworker/0:2+events10:37:000
169139910451042,3cyclictest12402-21kworker/3:0+events10:00:223
168979910451043,2cyclictest16325-21kworker/0:0+events12:33:280
168979910431041,2cyclictest14634-21kworker/0:1+events11:16:480
169139910411039,2cyclictest30123-21kworker/3:0+events10:18:493
168979910401038,2cyclictest4855-21kworker/0:0+events11:06:270
169139910391036,3cyclictest1422-21turbostat09:14:553
168979910391037,2cyclictest11984-21kworker/0:2+events10:30:440
169019910381037,1cyclictest16292-21kworker/1:0+events09:19:331
168979910371035,2cyclictest6834-21kworker/0:0+events08:03:300
168979910351033,2cyclictest2720-21kworker/0:1+events08:26:070
169139910271025,2cyclictest20407-21turbostat12:29:553
169079910161015,1cyclictest7226-21kworker/2:2+events12:17:432
169139910121010,2cyclictest20528-21turbostat11:54:553
169019910121011,1cyclictest30138-21kworker/1:1+events11:26:301
169139910031000,3cyclictest11123-21turbostat09:24:563
1691399999997,2cyclictest24839-21turbostat11:24:543
1690199999995,4cyclictest11167-21turbostat08:19:551
1690799994993,1cyclictest27792-21kworker/2:1+events07:35:432
1690199985983,2cyclictest6421-21turbostat11:04:541
1691399983981,2cyclictest5118-21turbostat08:04:563
1691399973971,2cyclictest29481-21turbostat11:29:553
1690199972971,1cyclictest27104-21kworker/1:2+events09:40:231
1690799969968,1cyclictest1026-21kworker/2:2+events09:13:132
1690799965964,1cyclictest27792-21kworker/2:1+events07:42:222
1690799960959,1cyclictest3987-21kworker/2:0+events11:52:212
1690799960959,1cyclictest28743-21kworker/2:0+events09:02:372
1690199955954,1cyclictest22286-21kworker/1:0+events09:38:401
1690799954951,1cyclictest30201-21kworker/2:1+events09:05:232
1689799954951,3cyclictest1676-21turbostat11:34:550
1690199953951,2cyclictest29407-21kworker/1:0+events12:38:141
1690799949948,1cyclictest20472-21kworker/2:1+events12:10:232
1690199939937,2cyclictest6940-21kworker/1:1+events09:29:521
1690199936935,1cyclictest30138-21kworker/1:1+events11:34:161
1690199925924,1cyclictest22286-21kworker/1:0+events09:31:061
1691399921919,2cyclictest15935-21turbostat11:49:553
1691399920918,2cyclictest6500-21turbostat09:19:563
1690199906904,2cyclictest25188-21turbostat10:14:561
1690199904901,3cyclictest11169-21turbostat12:19:541
1690199903901,2cyclictest20617-21turbostat10:09:551
1691399897895,2cyclictest24990-21turbostat12:34:543
1691399894892,2cyclictest29534-21turbostat09:44:563
1691399875873,2cyclictest27749-21turbostat08:59:553
1691399865863,2cyclictest29420-21turbostat07:44:553
1691399865863,2cyclictest1967-21turbostat12:09:553
1689799691690,1cyclictest11167-21turbostat08:19:550
1689799687684,3cyclictest6500-21turbostat09:19:550
1690799680679,1cyclictest24915-21turbostat07:34:552
1690199663661,2cyclictest1422-21turbostat09:14:551
1689799632629,3cyclictest29420-21turbostat07:44:550
1690199630626,4cyclictest11123-21turbostat09:24:561
1690199624623,1cyclictest24839-21turbostat11:24:541
1689799600596,4cyclictest11169-21turbostat12:19:540
1689799599596,3cyclictest20617-21turbostat10:09:550
1689799594591,3cyclictest2001-21turbostat10:24:560
1691399591586,2cyclictest10963-21turbostat09:59:563
1691399588587,1cyclictest25188-21turbostat10:14:563
1690799582579,3cyclictest15935-21turbostat11:49:552
1689799578576,2cyclictest20528-21turbostat11:54:550
1691399543540,3cyclictest9151-21turbostat08:14:553
1690799535533,2cyclictest27749-21turbostat08:59:552
1690199525521,4cyclictest24990-21turbostat12:34:551
1690799524522,2cyclictest1967-21turbostat12:09:552
1689799515514,1cyclictest11303-21turbostat11:44:540
1691399497494,3cyclictest1676-21turbostat11:34:553
1689799484483,1cyclictest29534-21turbostat09:44:560
16901991918,1cyclictest67750irq/123-lan12:08:511
16901991918,1cyclictest67750irq/123-lan12:00:161
16901991918,1cyclictest67750irq/123-lan11:52:541
16901991918,1cyclictest67750irq/123-lan11:05:221
16901991918,1cyclictest67750irq/123-lan10:34:071
16901991818,0cyclictest67750irq/123-lan11:39:331
16901991818,0cyclictest67750irq/123-lan09:58:071
16901991817,1cyclictest67750irq/123-lan12:20:161
16901991817,1cyclictest67750irq/123-lan11:41:181
16901991817,1cyclictest67750irq/123-lan10:50:131
16901991817,1cyclictest67750irq/123-lan10:43:451
16901991817,1cyclictest67750irq/123-lan10:00:211
16901991817,1cyclictest67750irq/123-lan09:47:551
16901991817,1cyclictest67750irq/123-lan08:40:111
16901991817,1cyclictest67750irq/123-lan08:39:181
16901991817,1cyclictest67750irq/123-lan07:55:121
16901991717,0cyclictest67750irq/123-lan12:29:251
16901991717,0cyclictest67750irq/123-lan09:05:101
16901991717,0cyclictest67750irq/123-lan08:20:561
16901991716,1cyclictest67750irq/123-lan11:18:541
16901991716,1cyclictest67750irq/123-lan11:10:091
16901991716,1cyclictest67750irq/123-lan10:15:451
16901991716,1cyclictest67750irq/123-lan08:55:171
16901991716,1cyclictest67750irq/123-lan08:50:121
16901991716,1cyclictest67750irq/123-lan08:45:211
16901991716,1cyclictest67750irq/123-lan08:25:191
16901991616,0cyclictest67750irq/123-lan11:59:111
16901991616,0cyclictest67750irq/123-lan10:49:181
16901991616,0cyclictest67750irq/123-lan10:36:371
16901991616,0cyclictest67750irq/123-lan09:54:081
16901991616,0cyclictest67750irq/123-lan09:02:141
16901991615,1cyclictest67750irq/123-lan12:10:151
16901991615,1cyclictest67750irq/123-lan11:45:101
16901991615,1cyclictest67750irq/123-lan10:55:161
16901991615,1cyclictest67750irq/123-lan08:00:111
16901991615,1cyclictest67750irq/123-lan07:50:131
16901991615,1cyclictest67750irq/123-lan07:45:151
16901991514,1cyclictest67750irq/123-lan10:20:121
16901991514,1cyclictest67750irq/123-lan08:14:181
16901991414,0cyclictest67750irq/123-lan07:40:131
16901991414,0cyclictest67750irq/123-lan07:32:191
16901991413,1cyclictest67750irq/123-lan10:25:171
16901991212,0cyclictest67750irq/123-lan08:05:161
16901991211,1cyclictest67750irq/123-lan08:30:151
1690799109,1cyclictest507-21systemd-resolve11:28:492
169139998,1cyclictest784-21snmpd09:08:073
169019999,0cyclictest67750irq/123-lan07:35:121
168979998,1cyclictest526-21avahi-daemon11:38:110
169139987,1cyclictest782-21lldpd10:53:083
169139987,1cyclictest526-21avahi-daemon09:38:043
169139987,1cyclictest526-21avahi-daemon08:25:213
169139987,1cyclictest507-21systemd-resolve12:01:023
169139986,2cyclictest784-21snmpd11:16:263
169139986,2cyclictest784-21snmpd11:11:303
169079988,0cyclictest784-21snmpd12:27:512
169079988,0cyclictest784-21snmpd10:45:492
169079988,0cyclictest784-21snmpd07:47:562
169079988,0cyclictest782-21lldpd07:10:062
169079987,1cyclictest782-21lldpd11:37:492
169079987,1cyclictest782-21lldpd11:07:322
169079987,1cyclictest507-21systemd-resolve12:22:542
169079986,2cyclictest784-21snmpd10:34:142
169079986,2cyclictest784-21snmpd10:04:252
169079986,2cyclictest784-21snmpd08:01:562
168979988,0cyclictest784-21snmpd09:12:430
168979988,0cyclictest782-21lldpd07:20:300
168979987,1cyclictest507-21systemd-resolve10:18:400
168979987,1cyclictest11158-21dump-pmu-power12:15:000
168979986,2cyclictest784-21snmpd09:25:140
169139977,0cyclictest782-21lldpd09:47:563
169139977,0cyclictest782-21lldpd08:48:283
169139976,1cyclictest784-21snmpd08:41:373
169139976,1cyclictest526-21avahi-daemon11:56:153
169139976,1cyclictest29422-21sh07:40:003
169139975,2cyclictest784-21snmpd12:20:393
169139975,2cyclictest784-21snmpd11:06:003
169139975,2cyclictest784-21snmpd09:51:003
169139974,3cyclictest30399-21cat12:35:133
169079977,0cyclictest784-21snmpd11:22:422
169079977,0cyclictest784-21snmpd10:25:522
169079977,0cyclictest784-21snmpd10:16:312
169079977,0cyclictest784-21snmpd07:52:022
169079977,0cyclictest782-21lldpd12:36:282
169079977,0cyclictest782-21lldpd10:05:242
169079977,0cyclictest782-21lldpd08:38:292
169079976,1cyclictest784-21snmpd11:03:082
169079976,1cyclictest784-21snmpd09:31:112
169079976,1cyclictest526-21avahi-daemon09:20:172
169079976,1cyclictest507-21systemd-resolve09:46:052
169079975,2cyclictest784-21snmpd11:43:422
169079973,3cyclictest9772-21ssh09:53:022
169079973,3cyclictest245-20systemd-journal12:30:462
169019977,0cyclictest784-21snmpd07:15:041
169019977,0cyclictest782-21lldpd07:10:341
169019975,2cyclictest784-21snmpd07:29:071
169019975,2cyclictest784-21snmpd07:24:011
168979977,0cyclictest784-21snmpd10:02:490
168979977,0cyclictest784-21snmpd07:28:580
168979977,0cyclictest782-21lldpd11:27:320
168979977,0cyclictest782-21lldpd09:30:180
168979977,0cyclictest782-21lldpd07:32:290
168979977,0cyclictest526-21avahi-daemon09:03:530
168979976,1cyclictest782-21lldpd07:37:570
168979974,3cyclictest784-21snmpd07:13:340
169139966,0cyclictest784-21snmpd09:32:083
169139966,0cyclictest782-21lldpd09:28:073
169139966,0cyclictest782-21lldpd07:57:543
169139965,1cyclictest26349-21tr10:45:183
169139964,2cyclictest784-21snmpd10:29:043
169139964,2cyclictest784-21snmpd08:34:453
169139963,3cyclictest547-21NetworkManager12:16:433
169139963,3cyclictest11871-21perl10:30:123
169139962,3cyclictest7683-21fw_conntrack08:05:133
169139962,3cyclictest547-21NetworkManager11:01:563
169139962,3cyclictest19033-21ssh10:37:433
169079966,0cyclictest784-21snmpd10:51:322
169079965,1cyclictest784-21snmpd10:59:302
169079965,1cyclictest784-21snmpd08:47:582
169079965,1cyclictest26062-21cpuspeed_turbos08:50:122
169079965,1cyclictest17987-21cpuspeed_turbos08:30:102
169079964,2cyclictest784-21snmpd10:35:442
169079964,2cyclictest784-21snmpd08:43:582
169079964,2cyclictest784-21snmpd08:10:262
169079964,2cyclictest547-21NetworkManager09:57:262
169079963,2cyclictest245-20systemd-journal11:15:272
169079963,2cyclictest245-20systemd-journal10:40:212
169079962,3cyclictest18654-21ssh09:27:102
169079962,3cyclictest16496-21irqstats11:10:142
168979962,3cyclictest29844-21ssh12:35:020
169139955,0cyclictest782-21lldpd12:10:373
169139955,0cyclictest782-21lldpd08:50:573
169139954,1cyclictest14314-21ntpq08:20:173
169139952,2cyclictest24940-21ssh10:44:493
169139952,2cyclictest2363-21ssh10:55:123
169139950,4cyclictest30808-21ntpq09:00:173
169079954,1cyclictest25657-21sh11:55:122
169079954,1cyclictest0-21swapper/209:40:142
169079953,2cyclictest784-21snmpd12:03:202
169079953,2cyclictest784-21snmpd07:20:062
169079953,2cyclictest762-21ntpd08:16:492
169079952,3cyclictest23322-21df_inode07:25:122
169079952,2cyclictest7972-21basename09:15:202
169079952,2cyclictest574-21rs:main2
169079952,2cyclictest4542-21ssh10:21:302
169079952,2cyclictest27033-21ssh09:35:492
169079952,2cyclictest2586-21ssh11:30:152
169079952,2cyclictest19457-21cut07:15:132
169079952,2cyclictest16725-21grep08:25:192
169079950,4cyclictest14211-21ntpq08:20:162
168979952,2cyclictest20451-21sed07:15:210
168979950,3cyclictest25244-21/usr/sbin/munin09:35:100
169139942,2cyclictest1781-21cat07:50:133
169139940,3cyclictest31884-21diskstats07:45:143
169139940,3cyclictest20220-21/usr/sbin/munin08:35:123
169079941,2cyclictest5148-21systemd-run08:00:002
169079940,3cyclictest7555-21df_inode08:05:132
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional