You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-06 - 21:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack8slot6s.osadl.org (updated Sun Sep 06, 2026 12:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
269089910531050,3cyclictest28500-21kworker/3:1+events09:34:023
269089910521050,2cyclictest8254-21kworker/3:2+events08:02:193
269089910521050,2cyclictest30553-21kworker/3:1+events07:20:443
269089910521050,2cyclictest16502-21kworker/3:0+events08:22:553
269089910521050,2cyclictest16502-21kworker/3:0+events08:07:283
269089910521049,3cyclictest24889-21kworker/3:2+events11:30:433
269089910511049,2cyclictest8254-21kworker/3:2+events07:50:443
269089910511049,2cyclictest8254-21kworker/3:2+events07:46:363
269089910511049,2cyclictest32630-21kworker/3:0+events07:36:063
269089910511049,2cyclictest30559-21kworker/3:2+events08:39:353
269089910511049,2cyclictest28500-21kworker/3:1+events08:31:493
269089910511049,2cyclictest27425-21kworker/3:1+events10:02:563
269089910511049,2cyclictest27248-21kworker/3:2+events09:41:143
269089910511049,2cyclictest20112-21kworker/3:0+events07:15:453
269089910511048,2cyclictest8254-21kworker/3:2+events07:58:473
268979910511049,2cyclictest2316-21kworker/0:0+events12:08:380
269089910501048,2cyclictest9994-21kworker/3:0+events09:57:223
269089910501048,2cyclictest27425-21kworker/3:1+events10:18:493
269089910501047,3cyclictest28507-21kworker/3:2+events07:12:443
269089910501047,3cyclictest27425-21kworker/3:1+events10:12:563
269089910501046,3cyclictest32630-21kworker/3:0+events07:44:553
269059910501048,2cyclictest4167-21kworker/2:2+events08:54:242
269059910501048,2cyclictest1098-21kworker/2:1+events07:30:212
269089910491047,2cyclictest24889-21kworker/3:2+events11:20:213
269089910491047,2cyclictest16502-21kworker/3:0+events08:26:053
269029910491048,1cyclictest12071-21kworker/1:0+events10:43:071
268979910491048,1cyclictest16865-21kworker/0:0+events10:36:470
269089910481046,2cyclictest28500-21kworker/3:1+events09:35:323
269089910481045,3cyclictest9994-21kworker/3:0+events10:06:273
269089910471045,2cyclictest24889-21kworker/3:2+events11:26:153
269059910461044,2cyclictest4167-21kworker/2:2+events08:57:332
268979910461044,2cyclictest16865-21kworker/0:0+events11:02:410
269089910451043,2cyclictest28500-21kworker/3:1+events09:25:223
269059910451043,2cyclictest4167-21kworker/2:2+events08:46:262
269089910441042,2cyclictest7393-21turbostat10:54:553
269059910441042,2cyclictest10793-21kworker/2:0+events09:16:292
269059910441042,2cyclictest10793-21kworker/2:0+events09:14:362
269059910441041,3cyclictest4167-21kworker/2:2+events09:05:222
269029910431042,1cyclictest18727-21kworker/1:0+events08:18:581
269059910421040,2cyclictest4167-21kworker/2:2+events09:01:282
269059910381036,2cyclictest30852-21kworker/2:0+events08:44:262
269059910371035,2cyclictest11642-21kworker/2:0+events07:28:222
269089910361034,2cyclictest28500-21kworker/3:1+events09:21:053
268979910341033,1cyclictest1477-21kworker/0:2+events11:42:300
269089910301028,2cyclictest9663-21kworker/3:0+events10:49:373
268979910291028,1cyclictest2316-21kworker/0:0+events12:02:550
269089910271024,3cyclictest27248-21kworker/3:2+events09:49:383
268979910121009,3cyclictest30531-21turbostat10:44:540
269089910041001,3cyclictest21499-21turbostat11:44:563
269089910031000,3cyclictest2929-21turbostat11:59:563
268979910011000,1cyclictest18804-21kworker/0:1+events11:09:050
26897991000997,3cyclictest20770-21turbostat08:19:540
2689799999998,1cyclictest2316-21kworker/0:0+events12:19:260
2689799996995,1cyclictest17231-21kworker/0:0+events11:45:220
2689799995994,1cyclictest16865-21kworker/0:0+events10:26:070
2689799985984,1cyclictest16865-21kworker/0:0+events10:31:300
2689799977975,2cyclictest16865-21kworker/0:0+events11:15:440
2689799976975,1cyclictest22659-21kworker/0:1+events09:50:230
2689799968967,1cyclictest7163-21kworker/0:2+events12:37:390
2690599961958,3cyclictest11680-21turbostat10:24:542
2689799959958,1cyclictest7163-21kworker/0:2+events12:10:440
2689799943942,1cyclictest7163-21kworker/0:2+events12:27:350
2689799937936,1cyclictest26790-21turbostat07:14:550
2690599935932,3cyclictest2647-21turbostat09:39:542
2690299927926,1cyclictest18727-21kworker/1:0+events08:11:161
2690899924921,3cyclictest25510-21turbostat10:39:543
2689799923923,0cyclictest18804-21kworker/0:1+events11:12:140
2690299920919,1cyclictest29072-21kworker/1:1+events10:23:161
2689799916915,1cyclictest16865-21kworker/0:0+events10:59:260
2689799916915,1cyclictest1477-21kworker/0:2+events11:53:040
2689799910909,1cyclictest2316-21kworker/0:0+events11:56:060
2689799909908,1cyclictest7163-21kworker/0:2+events12:34:540
2689799906905,1cyclictest7163-21kworker/0:2+events12:22:020
26902999040,903cyclictest17944-21vmstat09:15:221
26905999030,902cyclictest8645-21xargs10:15:202
2689799901900,1cyclictest17231-21kworker/0:0+events11:39:530
2689799900899,1cyclictest16865-21kworker/0:0+events10:51:470
2689799885883,2cyclictest28867-21turbostat08:39:540
2689799882880,2cyclictest2511-21turbostat07:34:550
2690899879876,3cyclictest11026-21turbostat09:14:553
2690599865862,3cyclictest20759-21turbostat09:24:542
2689799864862,2cyclictest16733-21turbostat08:09:540
2690299863861,2cyclictest26817-21turbostat12:24:561
2689799856854,2cyclictest3062-21turbostat11:24:560
2690599854852,2cyclictest30432-21turbostat11:19:552
2689799685684,1cyclictest12302-21turbostat11:34:550
2689799664663,1cyclictest2805-21turbostat10:49:560
2690899654651,3cyclictest16487-21turbostat09:54:553
2690899653650,3cyclictest8679-21turbostat12:39:563
2690899634630,4cyclictest30432-21turbostat11:19:553
2690299630626,4cyclictest16733-21turbostat08:09:541
2690899617616,1cyclictest15688-21turbostat09:19:553
2690299617614,3cyclictest7081-21turbostat10:19:541
2690899602601,1cyclictest2511-21turbostat07:34:553
2690599583580,3cyclictest28867-21turbostat08:39:552
2689799571570,1cyclictest11680-21turbostat10:24:540
2690899533530,3cyclictest30531-21turbostat10:44:553
2690299523519,4cyclictest25510-21turbostat10:39:541
2690899519518,1cyclictest20770-21turbostat08:19:553
2689799505504,1cyclictest11885-21turbostat09:49:540
2690599498495,3cyclictest30854-21turbostat07:24:562
26908991918,1cyclictest67750irq/123-lan12:10:003
26908991918,1cyclictest67750irq/123-lan10:55:153
26908991818,0cyclictest67750irq/123-lan11:12:273
26908991817,1cyclictest67750irq/123-lan12:30:123
26908991817,1cyclictest67750irq/123-lan12:20:163
26908991817,1cyclictest67750irq/123-lan11:50:113
26908991817,1cyclictest67750irq/123-lan10:25:143
26908991817,1cyclictest67750irq/123-lan10:20:483
26908991717,0cyclictest67750irq/123-lan08:10:103
26908991716,1cyclictest67750irq/123-lan12:26:243
26908991716,1cyclictest67750irq/123-lan12:00:143
26908991716,1cyclictest67750irq/123-lan11:45:143
26908991716,1cyclictest67750irq/123-lan11:36:123
26908991716,1cyclictest67750irq/123-lan11:01:473
26908991716,1cyclictest67750irq/123-lan08:50:153
26908991616,0cyclictest67750irq/123-lan12:10:113
26908991616,0cyclictest67750irq/123-lan11:06:083
26908991616,0cyclictest67750irq/123-lan08:59:133
26908991615,1cyclictest67750irq/123-lan12:15:213
26908991515,0cyclictest67750irq/123-lan10:31:353
26908991514,1cyclictest67750irq/123-lan09:09:093
26908991514,1cyclictest67750irq/123-lan09:00:083
26908991514,1cyclictest67750irq/123-lan08:45:203
26908991414,0cyclictest67750irq/123-lan07:29:083
26897991210,2cyclictest784-21snmpd08:23:500
26908991110,1cyclictest67750irq/123-lan08:43:093
269059998,1cyclictest784-21snmpd11:56:452
269059998,1cyclictest784-21snmpd10:54:212
269059998,1cyclictest784-21snmpd08:14:522
269059998,1cyclictest784-21snmpd08:01:432
269029998,1cyclictest784-21snmpd11:21:051
269029998,1cyclictest784-21snmpd08:47:141
269059988,0cyclictest782-21lldpd09:55:292
269059988,0cyclictest782-21lldpd07:10:072
269059987,1cyclictest784-21snmpd10:09:472
269059987,1cyclictest12551-21fschecks_count10:55:112
269059986,2cyclictest784-21snmpd12:28:332
269029988,0cyclictest784-21snmpd07:49:031
269029988,0cyclictest784-21snmpd07:39:511
269029988,0cyclictest782-21lldpd09:38:571
269029988,0cyclictest782-21lldpd09:05:421
269029988,0cyclictest782-21lldpd08:40:591
269029987,1cyclictest784-21snmpd11:03:281
269029987,1cyclictest782-21lldpd07:30:551
269029987,1cyclictest526-21avahi-daemon12:17:491
269029986,2cyclictest784-21snmpd12:28:421
269029986,2cyclictest784-21snmpd11:25:081
269029986,2cyclictest784-21snmpd10:57:061
268979988,0cyclictest784-21snmpd10:07:170
268979988,0cyclictest782-21lldpd10:19:230
268979987,1cyclictest526-21avahi-daemon09:34:390
268979986,2cyclictest784-21snmpd09:27:240
268979986,2cyclictest784-21snmpd09:11:440
268979986,2cyclictest3463-21cat09:35:140
269059977,0cyclictest784-21snmpd11:21:262
269059977,0cyclictest784-21snmpd07:55:392
269059977,0cyclictest782-21lldpd12:22:102
269059977,0cyclictest782-21lldpd11:02:592
269059977,0cyclictest782-21lldpd10:02:092
269059977,0cyclictest782-21lldpd08:32:002
269059976,1cyclictest526-21avahi-daemon11:29:282
269059976,1cyclictest526-21avahi-daemon10:43:322
269059976,1cyclictest21773-21latency_hist12:15:002
269059973,3cyclictest15742-21ssh12:08:202
269029977,0cyclictest784-21snmpd11:54:501
269029977,0cyclictest784-21snmpd10:06:261
269029977,0cyclictest784-21snmpd07:58:421
269029977,0cyclictest782-21lldpd09:55:551
269029977,0cyclictest782-21lldpd09:50:201
269029977,0cyclictest782-21lldpd09:01:191
269029977,0cyclictest17963-21needreboot11:35:151
269029976,1cyclictest784-21snmpd11:17:511
269029976,1cyclictest784-21snmpd08:23:471
269029976,1cyclictest526-21avahi-daemon12:37:111
269029976,1cyclictest526-21avahi-daemon10:49:441
269029976,1cyclictest21437-21fschecks_time09:20:131
269029976,1cyclictest13293-21ntp_kernel_err12:05:161
269029975,2cyclictest784-21snmpd09:44:211
269029975,2cyclictest784-21snmpd08:27:581
269029975,2cyclictest784-21snmpd07:40:161
269029975,2cyclictest784-21snmpd07:10:381
268979977,0cyclictest784-21snmpd11:26:410
268979977,0cyclictest784-21snmpd09:07:020
268979977,0cyclictest784-21snmpd07:54:330
268979977,0cyclictest782-21lldpd07:23:350
268979977,0cyclictest25384-21sh09:24:590
268979976,1cyclictest784-21snmpd07:39:090
268979976,1cyclictest782-21lldpd07:40:400
268979976,1cyclictest507-21systemd-resolve10:13:350
268979976,1cyclictest25638-21sendmail09:59:590
268979975,2cyclictest784-21snmpd09:40:270
269059966,0cyclictest782-21lldpd08:05:312
269059966,0cyclictest782-21lldpd07:40:282
269059966,0cyclictest782-21lldpd07:38:492
269059966,0cyclictest782-21lldpd07:19:592
269059966,0cyclictest0-21swapper/210:48:382
269059965,1cyclictest784-21snmpd12:34:132
269059965,1cyclictest784-21snmpd09:27:152
269059965,1cyclictest784-21snmpd08:20:022
269059965,1cyclictest526-21avahi-daemon09:46:052
269059965,1cyclictest507-21systemd-resolve11:05:182
269059964,2cyclictest784-21snmpd10:12:262
269059963,2cyclictest29279-21ssh11:13:052
269059962,3cyclictest9683-21ssh09:41:182
269059962,3cyclictest2806-21ssh11:54:502
269059962,3cyclictest24804-21ssh10:33:512
269059962,3cyclictest18891-21ssh11:35:382
269029966,0cyclictest784-21snmpd09:12:451
269029966,0cyclictest784-21snmpd08:58:331
269029966,0cyclictest782-21lldpd12:10:051
269029966,0cyclictest782-21lldpd09:31:141
269029966,0cyclictest782-21lldpd07:23:001
269029966,0cyclictest526-21avahi-daemon08:39:541
269029966,0cyclictest526-21avahi-daemon07:29:091
269029965,1cyclictest784-21snmpd10:54:521
269029965,1cyclictest784-21snmpd08:53:051
269029965,1cyclictest526-21avahi-daemon11:34:531
269029965,1cyclictest526-21avahi-daemon11:11:411
269029965,1cyclictest331ktimers/109:27:291
269029964,2cyclictest8249-21grep12:00:141
269029964,2cyclictest784-21snmpd12:31:541
269029964,2cyclictest784-21snmpd10:13:081
269029963,3cyclictest547-21NetworkManager10:00:381
269029963,2cyclictest24241-21ssh10:32:541
269029962,4cyclictest547-21NetworkManager11:05:121
269029962,3cyclictest3252-21ssh11:55:121
269029962,3cyclictest26482-21apt-get11:45:111
269029961,5cyclictest547-21NetworkManager08:02:311
268979966,0cyclictest784-21snmpd09:01:270
268979966,0cyclictest784-21snmpd07:19:470
268979966,0cyclictest782-21lldpd08:55:010
268979966,0cyclictest782-21lldpd08:13:310
268979966,0cyclictest782-21lldpd08:03:380
268979966,0cyclictest782-21lldpd07:46:200
268979966,0cyclictest526-21avahi-daemon08:52:050
268979965,1cyclictest784-21snmpd07:57:120
268979964,2cyclictest784-21snmpd08:33:310
268979964,2cyclictest784-21snmpd07:28:240
268979963,2cyclictest18980-21ssh09:17:030
269059955,0cyclictest784-21snmpd12:16:462
269059955,0cyclictest13034-21cut11:30:132
269059954,1cyclictest421ktimers/210:28:272
269059953,2cyclictest547-21NetworkManager07:52:102
269059953,1cyclictest25513-21apt-get10:35:002
269059952,3cyclictest547-21NetworkManager08:15:022
269059952,2cyclictest9760-21sh12:01:002
269059952,2cyclictest245-20systemd-journal11:47:032
269059952,2cyclictest21806-21ssh11:40:062
269059952,2cyclictest1020-21ssh09:32:172
269059951,3cyclictest17031-21df_inode09:50:132
269059951,3cyclictest11848-21ssh12:37:372
269029953,2cyclictest547-21NetworkManager09:45:591
269029953,2cyclictest245-20systemd-journal10:26:571
269029952,2cyclictest29809-21ntpq07:15:161
269029952,2cyclictest27407-21grep08:30:121
269029952,2cyclictest245-20systemd-journal07:50:241
269029952,2cyclictest22647-21nfs4_client11:40:171
268979952,2cyclictest27698-21ssh10:00:380
269059941,2cyclictest9518-21wc07:45:152
269059940,4cyclictest25855-21ntpq08:25:162
268979944,0cyclictest0-21swapper/008:40:150
268979940,3cyclictest25253-21diskstats08:25:120
268979940,3cyclictest1518-21ntpq08:45:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional