You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-19 - 23:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Sat Sep 19, 2026 12:43:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
189339910541050,2cyclictest23337-21kworker/1:1+events09:33:181
189339910541049,3cyclictest23830-21kworker/1:2+events09:40:421
189339910531049,3cyclictest7370-21turbostat11:09:551
189339910531049,2cyclictest5223-21kworker/1:1+events10:28:271
189339910531049,2cyclictest31338-21kworker/1:2+events11:02:011
189339910531049,2cyclictest11126-21kworker/1:2+events09:59:151
189339910521049,2cyclictest17382-21kworker/1:0+events09:28:291
189389910511049,2cyclictest29280-21kworker/2:0+events10:04:532
189389910511048,3cyclictest32375-21turbostat12:14:552
189339910511049,2cyclictest31338-21kworker/1:2+events10:55:451
189339910511049,2cyclictest23337-21kworker/1:1+events09:35:441
189339910511049,2cyclictest23019-21kworker/1:0+events07:16:051
189279910501049,1cyclictest30467-21kworker/0:0+events08:55:430
189389910491047,2cyclictest19830-21kworker/2:1+events11:13:592
189389910491047,2cyclictest19830-21kworker/2:1+events10:07:412
189339910481046,2cyclictest23019-21kworker/1:0+events07:14:301
189389910441042,2cyclictest19830-21kworker/2:1+events11:16:242
189389910431041,2cyclictest19830-21kworker/2:1+events10:20:312
189279910411040,1cyclictest18274-21kworker/0:0+events10:49:520
189429910381036,2cyclictest9762-21turbostat11:49:543
189389910381035,3cyclictest20388-21turbostat11:24:542
189389910351033,2cyclictest25744-21kworker/2:0+events10:13:192
189389910351032,3cyclictest19830-21kworker/2:1+events10:15:232
189339910311028,3cyclictest13562-21turbostat10:39:551
189279910301027,3cyclictest21723-21turbostat08:39:550
189279910301026,4cyclictest8640-21turbostat12:24:560
189279910211020,1cyclictest16950-21kworker/0:2+events10:42:220
189429910161011,4cyclictest22874-21kworker/3:1+events07:32:013
189429910141014,0cyclictest25862-21kworker/3:1+events08:45:223
189429910141013,1cyclictest25196-21kworker/3:0+events07:27:553
189279910121011,1cyclictest7306-21kworker/0:0+events11:53:580
189429910091007,2cyclictest19602-21kworker/3:1+events12:31:353
189429910081006,2cyclictest3876-21kworker/3:0+events09:11:293
189429910081006,0cyclictest3876-21kworker/3:0+events09:47:533
189279910081004,2cyclictest7306-21kworker/0:0+events11:25:200
189429910061004,2cyclictest13886-21kworker/3:1+events11:36:463
189339910031000,3cyclictest5394-21turbostat11:44:541
189429910021001,1cyclictest3876-21kworker/3:0+events09:50:433
189429910021001,1cyclictest28596-21kworker/3:2+events08:30:223
189429910021000,2cyclictest23736-21turbostat08:44:553
189279910021001,1cyclictest14455-21kworker/0:1+events12:04:460
189429910011000,1cyclictest3876-21kworker/3:0+events09:16:053
18942991000999,1cyclictest32614-21kworker/3:0+events11:40:433
18942991000999,1cyclictest1523-21kworker/3:2+events07:21:063
1894299999998,1cyclictest30639-21kworker/3:0+events07:47:593
1894299999998,1cyclictest19602-21kworker/3:1+events12:26:383
1892799998997,1cyclictest20744-21kworker/0:2+events11:22:320
1892799998996,2cyclictest23572-21kworker/0:2+events12:06:060
1892799998994,4cyclictest12651-21turbostat09:24:550
1894299997996,1cyclictest3876-21kworker/3:0+events10:34:453
1894299997996,1cyclictest28596-21kworker/3:2+events08:50:223
1894299997996,1cyclictest28596-21kworker/3:2+events07:54:103
1894299996995,1cyclictest9772-21kworker/3:2+events09:20:543
1894299996995,1cyclictest9171-21kworker/3:2+events10:36:453
1894299996995,1cyclictest28596-21kworker/3:2+events07:36:073
1894299996995,1cyclictest23374-21kworker/3:0+events08:38:213
1894299995994,1cyclictest28596-21kworker/3:2+events07:42:573
1894299994993,1cyclictest28596-21kworker/3:2+events08:04:123
1894299992990,2cyclictest19602-21kworker/3:1+events12:38:183
1894299992990,1cyclictest19602-21kworker/3:1+events11:57:093
1892799990988,2cyclictest26680-21turbostat10:54:550
1894299989988,1cyclictest6351-21kworker/3:1+events07:57:003
1893399989987,2cyclictest13027-21turbostat12:29:551
1892799988986,2cyclictest4909-21turbostat10:29:550
1894299987984,3cyclictest13203-21turbostat08:19:553
1892799986985,1cyclictest7306-21kworker/0:0+events11:31:270
1893899984981,3cyclictest28214-21turbostat08:54:552
1894299965963,2cyclictest2969-21turbostat11:04:553
1893399964961,3cyclictest17262-21turbostat08:29:551
1893899963961,2cyclictest20882-21turbostat07:19:552
1892799962961,1cyclictest12172-21kworker/0:0+events08:07:340
1892799959958,1cyclictest28776-21kworker/0:1+events11:06:410
1892799957955,1cyclictest14880-21kworker/0:1+events08:20:230
1892799957954,3cyclictest4715-21turbostat07:59:550
1892799956955,1cyclictest28400-21kworker/0:0+events12:13:060
1892799955954,1cyclictest12172-21kworker/0:0+events08:15:220
1892799954953,1cyclictest23572-21kworker/0:2+events12:15:220
1892799949948,1cyclictest12172-21kworker/0:0+events08:12:130
1892799944943,1cyclictest16950-21kworker/0:2+events09:04:110
1892799944943,1cyclictest1518-21kworker/0:1+events09:09:000
1892799943942,1cyclictest19013-21kworker/0:1+events08:27:210
1892799932929,3cyclictest30112-21turbostat09:44:550
1894299915913,2cyclictest17039-21turbostat09:29:553
1894299907903,3cyclictest16017-21turbostat11:19:543
1892799907906,1cyclictest8277-21turbostat09:19:550
1894299864862,2cyclictest32484-21turbostat10:24:553
1893899854852,2cyclictest18855-21turbostat11:59:552
1893399693690,3cyclictest32484-21turbostat10:24:551
1894299687682,3cyclictest14516-21turbostat11:54:553
1893399685682,3cyclictest6454-21turbostat09:54:551
1892799673670,3cyclictest18855-21turbostat11:59:550
1892799672669,3cyclictest2969-21turbostat11:04:550
1893399671670,1cyclictest12651-21turbostat09:24:551
1892799669666,3cyclictest6721-21turbostat08:04:550
1893399667664,3cyclictest26680-21turbostat10:54:551
1893899663660,3cyclictest10813-21turbostat09:59:552
1892799663661,2cyclictest13562-21turbostat10:39:550
1894299645643,2cyclictest17262-21turbostat08:29:553
1894299633629,3cyclictest20882-21turbostat07:19:553
1893899622619,3cyclictest7370-21turbostat11:09:552
1892799609608,1cyclictest16017-21turbostat11:19:540
1892799603599,4cyclictest28214-21turbostat08:54:550
1894299591590,1cyclictest4909-21turbostat10:29:553
1894299557554,3cyclictest29111-21turbostat11:34:543
1892799543541,2cyclictest9762-21turbostat11:49:550
1894299533532,1cyclictest30112-21turbostat09:44:553
1894299526525,1cyclictest1880-21turbostat09:09:553
1894299451450,1cyclictest8640-21turbostat12:24:563
18933991918,1cyclictest67750irq/123-lan12:32:071
18933991918,1cyclictest67750irq/123-lan12:00:141
18933991918,1cyclictest67750irq/123-lan11:51:001
18933991918,1cyclictest67750irq/123-lan11:30:201
18933991918,1cyclictest67750irq/123-lan11:20:171
18933991918,1cyclictest67750irq/123-lan10:15:171
18933991818,0cyclictest67750irq/123-lan12:23:131
18933991818,0cyclictest67750irq/123-lan12:10:181
18933991818,0cyclictest67750irq/123-lan11:26:411
18933991818,0cyclictest67750irq/123-lan11:15:121
18933991818,0cyclictest67750irq/123-lan11:10:201
18933991817,1cyclictest67750irq/123-lan12:36:201
18933991817,1cyclictest67750irq/123-lan12:17:001
18933991817,1cyclictest67750irq/123-lan12:05:161
18933991817,1cyclictest67750irq/123-lan11:58:281
18933991817,1cyclictest67750irq/123-lan11:46:291
18933991817,1cyclictest67750irq/123-lan10:40:411
18933991817,1cyclictest67750irq/123-lan10:33:571
18933991817,1cyclictest67750irq/123-lan09:45:131
18933991817,1cyclictest67750irq/123-lan09:17:031
18933991817,1cyclictest67750irq/123-lan08:09:091
18933991817,1cyclictest67750irq/123-lan08:01:171
18933991717,0cyclictest67750irq/123-lan09:13:141
18933991717,0cyclictest67750irq/123-lan08:10:131
18933991716,1cyclictest67750irq/123-lan11:38:351
18933991716,1cyclictest67750irq/123-lan10:14:401
18933991716,1cyclictest67750irq/123-lan07:30:191
18933991716,1cyclictest67750irq/123-lan07:25:121
18933991616,0cyclictest67750irq/123-lan10:05:181
18933991616,0cyclictest67750irq/123-lan10:02:571
18933991616,0cyclictest67750irq/123-lan08:30:191
18933991616,0cyclictest67750irq/123-lan07:49:061
18933991615,1cyclictest67750irq/123-lan10:45:171
18933991615,1cyclictest67750irq/123-lan08:55:001
18933991615,1cyclictest67750irq/123-lan08:49:191
18933991615,1cyclictest67750irq/123-lan08:44:121
18933991515,0cyclictest67750irq/123-lan09:01:071
18933991515,0cyclictest67750irq/123-lan08:15:141
18933991515,0cyclictest67750irq/123-lan07:54:491
18933991514,1cyclictest67750irq/123-lan07:55:141
18933991514,1cyclictest67750irq/123-lan07:20:181
18933991413,1cyclictest67750irq/123-lan09:07:531
18933991313,0cyclictest67750irq/123-lan08:39:491
18933991313,0cyclictest67750irq/123-lan08:20:201
18933991212,0cyclictest67750irq/123-lan08:57:191
18942991010,0cyclictest784-21snmpd12:06:153
189429999,0cyclictest784-21snmpd11:23:043
189429999,0cyclictest784-21snmpd11:11:013
189429999,0cyclictest784-21snmpd09:56:013
189429999,0cyclictest784-21snmpd08:22:283
189429999,0cyclictest784-21snmpd08:10:043
189429998,1cyclictest784-21snmpd07:14:143
189429998,1cyclictest526-21avahi-daemon10:06:533
189389998,1cyclictest9252-21fschecks_count12:20:142
189389998,1cyclictest784-21snmpd09:47:522
189389998,1cyclictest784-21snmpd09:40:222
189339999,0cyclictest67750irq/123-lan07:35:131
189429988,0cyclictest784-21snmpd08:05:523
189429987,1cyclictest784-21snmpd09:03:363
189429980,8cyclictest32386-21turbostat12:14:553
189389988,0cyclictest782-21lldpd12:35:042
189389988,0cyclictest782-21lldpd12:18:562
189389988,0cyclictest782-21lldpd12:01:272
189389988,0cyclictest782-21lldpd11:50:162
189389988,0cyclictest782-21lldpd11:00:272
189389988,0cyclictest782-21lldpd10:37:062
189389988,0cyclictest782-21lldpd07:56:002
189389987,1cyclictest784-21snmpd10:57:302
189389987,1cyclictest784-21snmpd10:32:032
189389987,1cyclictest784-21snmpd09:12:392
189389987,1cyclictest782-21lldpd12:34:022
189389986,2cyclictest784-21snmpd12:29:012
189339987,1cyclictest67750irq/123-lan07:40:121
189279988,0cyclictest784-21snmpd07:46:360
189279988,0cyclictest782-21lldpd12:38:280
189279987,1cyclictest784-21snmpd07:17:260
189279987,1cyclictest526-21avahi-daemon11:11:450
189279987,1cyclictest526-21avahi-daemon07:20:010
189429977,0cyclictest784-21snmpd10:58:493
189429977,0cyclictest784-21snmpd09:30:183
189429977,0cyclictest782-21lldpd10:46:393
189429976,1cyclictest784-21snmpd10:50:283
189429976,1cyclictest762-21ntpd11:26:273
189389977,0cyclictest784-21snmpd07:40:152
189389977,0cyclictest782-21lldpd09:05:412
189389977,0cyclictest245-20systemd-journal10:25:112
189389976,1cyclictest526-21avahi-daemon08:03:202
189389976,1cyclictest22706-21nvmesmart_nvme009:30:192
189389975,2cyclictest784-21snmpd09:03:242
189389973,3cyclictest245-20systemd-journal10:54:142
189279977,0cyclictest784-21snmpd11:35:020
189279977,0cyclictest784-21snmpd08:41:560
189279976,1cyclictest784-21snmpd10:20:450
189279976,1cyclictest784-21snmpd10:19:380
189279976,1cyclictest762-21ntpd09:12:440
189279976,1cyclictest15626-21cpuspeed_turbos10:00:140
189279975,2cyclictest784-21snmpd10:59:320
189279975,2cyclictest784-21snmpd09:32:490
189279975,2cyclictest784-21snmpd08:49:170
189279975,2cyclictest19570-21cat07:10:130
189429966,0cyclictest784-21snmpd12:04:073
189429966,0cyclictest784-21snmpd09:39:163
189429966,0cyclictest782-21lldpd10:15:463
189429965,1cyclictest784-21snmpd12:19:453
189429965,1cyclictest507-21systemd-resolve10:12:293
189429962,3cyclictest245-20systemd-journal10:00:213
189389966,0cyclictest784-21snmpd09:25:092
189389966,0cyclictest784-21snmpd07:11:172
189389966,0cyclictest782-21lldpd11:29:342
189389966,0cyclictest782-21lldpd08:37:272
189389965,1cyclictest782-21lldpd07:21:162
189389965,1cyclictest3382-21cat07:50:162
189389964,2cyclictest784-21snmpd07:25:532
189389963,3cyclictest547-21NetworkManager09:15:382
189389963,3cyclictest18334-21ntpq08:25:172
189389962,3cyclictest280-21systemd-udevd09:37:302
189279965,1cyclictest526-21avahi-daemon09:58:530
189279965,1cyclictest29301-21cpuspeed_turbos07:35:120
189279963,2cyclictest7300-21ssh11:40:270
189279963,2cyclictest24226-21ssh10:10:130
189279963,2cyclictest19375-21ssh12:30:440
189279962,3cyclictest7892-21mailstats09:50:180
189279962,3cyclictest5006-21ssh09:47:230
189279962,3cyclictest28696-21ssh09:37:250
189429955,0cyclictest784-21snmpd10:42:363
189429954,1cyclictest507-21systemd-resolve11:05:123
189389954,1cyclictest784-21snmpd07:37:242
189389954,1cyclictest421ktimers/212:05:532
189389953,2cyclictest784-21snmpd08:21:522
189389952,3cyclictest7099-21perl09:50:122
189389952,3cyclictest547-21NetworkManager11:32:042
189389952,3cyclictest11120-21seq08:09:202
189389952,2cyclictest7426-21ssh11:40:412
189389952,2cyclictest5180-21ssh11:39:392
189389952,2cyclictest245-20systemd-journal09:22:572
189389952,2cyclictest21888-21sed08:35:002
189389952,2cyclictest20090-21sh10:40:562
189389952,2cyclictest13623-21rm11:48:202
189389952,2cyclictest11572-21tr08:10:132
189279955,0cyclictest782-21lldpd07:27:500
189279953,2cyclictest784-21snmpd08:32:550
189279953,2cyclictest3380-21cat07:50:160
189279953,2cyclictest27500-21perl07:30:130
189279953,0cyclictest0-21swapper/012:26:220
189279952,2cyclictest20691-21sed10:05:160
189279952,2cyclictest19511-21ssh09:26:330
189279952,2cyclictest11681-21ssh10:31:330
189429940,4cyclictest31326-21ntpq08:55:193
189389942,2cyclictest547-21NetworkManager07:34:472
189389941,2cyclictest26860-21hddtemp_smartct08:45:152
189389940,3cyclictest31296-21ntpq08:55:192
189389940,3cyclictest25265-21ntpq08:40:192
189389940,3cyclictest245-20systemd-journal07:46:072
189389940,3cyclictest14306-21ntp_states08:15:172
189279941,2cyclictest31028-21latency_hist07:40:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional