You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-05 - 20:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack8slot6s.osadl.org (updated Sat Sep 05, 2026 12:43:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
283619910531050,2cyclictest1958-21kworker/3:2+events10:57:323
283619910521050,2cyclictest4383-21kworker/3:2+events09:04:193
283619910511049,2cyclictest1958-21kworker/3:2+events10:42:523
283619910511049,2cyclictest1958-21kworker/3:2+events10:42:523
283619910491046,3cyclictest25420-21kworker/3:0+events11:07:243
283449910481046,2cyclictest8894-21kworker/0:2+events07:19:240
283449910471045,2cyclictest18969-21turbostat12:14:560
283449910341033,1cyclictest27945-21kworker/0:2+events11:13:580
283449910331032,1cyclictest27897-21kworker/0:0+events09:06:070
283619910311030,1cyclictest30391-21kworker/3:0+events08:46:153
283619910311028,3cyclictest17917-21turbostat09:54:553
283619910301028,2cyclictest14372-21turbostat12:09:553
283569910301028,2cyclictest32215-21turbostat11:19:552
283619910281024,4cyclictest27610-21turbostat11:14:563
283529910271025,2cyclictest7360-21kworker/1:1+events11:58:061
283529910211019,2cyclictest7360-21kworker/1:1+events10:24:561
283529910211019,2cyclictest7360-21kworker/1:1+events10:19:081
283619910201019,1cyclictest24665-21kworker/3:2+events08:25:443
283529910201019,1cyclictest23254-21kworker/1:0+events10:14:001
283529910201019,1cyclictest18360-21kworker/1:1+events08:09:011
283529910201018,2cyclictest8447-21kworker/1:0+events07:46:501
283529910201018,2cyclictest23254-21kworker/1:0+events10:03:171
283449910201019,1cyclictest2257-21kworker/0:0+events12:27:430
283529910191015,2cyclictest24576-21kworker/1:2+events10:09:121
283449910191018,1cyclictest14456-21kworker/0:2+events09:11:050
283449910181017,1cyclictest2082-21kworker/0:1+events11:38:410
283529910171015,2cyclictest8447-21kworker/1:0+events07:55:441
283449910171016,1cyclictest8894-21kworker/0:2+events07:31:040
283449910171016,1cyclictest24250-21kworker/0:2+events09:29:540
283449910171016,1cyclictest24250-21kworker/0:2+events09:21:520
283449910161016,0cyclictest9380-21kworker/0:0+events11:30:430
283529910151013,2cyclictest8447-21kworker/1:0+events08:04:451
283529910151013,2cyclictest22377-21kworker/1:1+events07:44:451
283529910141012,2cyclictest8447-21kworker/1:0+events07:53:501
283529910141011,3cyclictest8447-21kworker/1:0+events08:12:351
283449910121011,1cyclictest7616-21kworker/0:1+events12:31:290
283529910091007,2cyclictest27843-21kworker/1:2+events11:47:131
283529910081004,2cyclictest23254-21kworker/1:0+events09:59:281
283449910081005,2cyclictest8894-21kworker/0:2+events07:10:220
283529910071003,2cyclictest8447-21kworker/1:0+events08:17:451
283529910071003,2cyclictest12244-21kworker/1:2+events10:26:501
283449910071006,1cyclictest9380-21kworker/0:0+events11:26:250
283449910071006,1cyclictest8894-21kworker/0:2+events07:25:210
283449910061005,1cyclictest27897-21kworker/0:0+events09:16:370
283449910061003,3cyclictest23032-21turbostat11:09:550
283449910061002,1cyclictest29564-21kworker/0:0+events09:31:030
283449910051005,0cyclictest29564-21kworker/0:0+events09:36:160
283449910051004,1cyclictest2082-21kworker/0:1+events12:20:520
283449910031001,2cyclictest10047-21turbostat12:39:550
28361991002999,3cyclictest18254-21turbostat11:39:543
283449910011000,1cyclictest8347-21kworker/0:1+events07:39:110
28361991000998,2cyclictest9656-21kworker/3:2+events12:12:213
2835699999996,3cyclictest30291-21turbostat07:19:552
2836199997995,2cyclictest22782-21kworker/3:0+events11:52:223
2835699993992,1cyclictest4537-21kworker/2:0+events10:45:452
2836199992991,1cyclictest14942-21kworker/3:1+events09:45:433
2835699992990,2cyclictest5128-21turbostat11:59:552
2836199990989,1cyclictest830-21kworker/3:1+events08:31:273
2836199986983,3cyclictest32439-21kworker/3:1+events12:00:213
2836199985983,2cyclictest22855-21turbostat11:44:543
2834499978976,2cyclictest32358-21turbostat10:44:550
2834499978976,2cyclictest32358-21turbostat10:44:550
2836199973972,1cyclictest28119-21kworker/3:0+events10:37:123
2835299971969,2cyclictest4600-21turbostat10:49:551
2836199969968,1cyclictest8176-21kworker/3:0+events09:40:433
2836199964963,1cyclictest8079-21kworker/3:1+events08:57:303
2835699961959,2cyclictest8256-21turbostat09:44:552
2835299960957,3cyclictest5020-21turbostat12:34:561
2834499959957,2cyclictest32767-21turbostat08:44:550
2834499953949,4cyclictest18599-21turbostat10:29:550
2835299944941,3cyclictest12502-21turbostat09:14:551
2836199943941,2cyclictest9656-21kworker/3:2+events12:35:223
2836199939937,2cyclictest6401-21turbostat07:39:553
2836199932930,2cyclictest28027-21kworker/3:0+events07:23:503
2835699928925,3cyclictest27914-21turbostat11:49:542
2836199908907,1cyclictest4383-21kworker/3:2+events08:50:223
2836199908906,2cyclictest1958-21kworker/3:2+events10:51:503
2834499908904,3cyclictest22675-21turbostat08:19:550
2836199905904,1cyclictest6387-21kworker/3:0+events11:21:493
2836199903901,1cyclictest9656-21kworker/3:2+events12:16:153
2836199900899,1cyclictest1958-21kworker/3:2+events11:01:063
2836199899898,1cyclictest830-21kworker/3:1+events08:44:193
2836199899898,1cyclictest20088-21kworker/3:2+events10:34:473
2836199898897,1cyclictest30391-21kworker/3:0+events08:38:313
2836199893893,0cyclictest32547-21kworker/3:1+events11:15:433
2836199887885,2cyclictest24665-21kworker/3:2+events08:24:463
2834499887885,2cyclictest6441-21turbostat08:59:550
2836199855853,2cyclictest12590-21turbostat07:54:553
2835299703702,1cyclictest17917-21turbostat09:54:551
2835299659657,2cyclictest22855-21turbostat11:44:541
2836199647646,1cyclictest30291-21turbostat07:19:553
2836199644640,4cyclictest5128-21turbostat11:59:553
2836199614611,3cyclictest3646-21turbostat09:39:553
2835299614612,2cyclictest6401-21turbostat07:39:551
2836199606603,3cyclictest4600-21turbostat10:49:553
2836199593591,2cyclictest5020-21turbostat12:34:553
2834499579578,1cyclictest8443-21turbostat09:04:550
2836199576575,1cyclictest27914-21turbostat11:49:543
2835299570567,3cyclictest10047-21turbostat12:39:551
2834499564560,3cyclictest4399-21turbostat11:24:540
2836199553549,3cyclictest22675-21turbostat08:19:553
2835299550549,1cyclictest32501-21turbostat11:54:551
2834499545543,2cyclictest32308-21turbostat07:24:550
2835699528526,2cyclictest32358-21turbostat10:44:552
2835699528526,2cyclictest32358-21turbostat10:44:552
2834499528525,3cyclictest23550-21turbostat12:19:550
2836199439436,2cyclictest18599-21turbostat10:29:553
28361991918,1cyclictest67750irq/123-lan09:22:403
28361991918,1cyclictest67750irq/123-lan08:10:213
28361991818,0cyclictest67750irq/123-lan09:55:163
28361991817,1cyclictest67750irq/123-lan10:15:173
28361991817,1cyclictest67750irq/123-lan10:00:193
28361991817,1cyclictest67750irq/123-lan09:32:073
28361991817,1cyclictest67750irq/123-lan09:10:153
28356991818,0cyclictest784-21snmpd07:34:572
28361991716,1cyclictest67750irq/123-lan12:25:183
28361991716,1cyclictest67750irq/123-lan10:20:123
28361991716,1cyclictest67750irq/123-lan10:10:163
28361991716,1cyclictest67750irq/123-lan09:15:543
28361991616,0cyclictest67750irq/123-lan09:25:123
28361991615,1cyclictest67750irq/123-lan12:20:473
28361991615,1cyclictest67750irq/123-lan11:31:343
28361991615,1cyclictest67750irq/123-lan08:05:213
28361991615,1cyclictest67750irq/123-lan07:25:153
28361991515,0cyclictest67750irq/123-lan07:45:103
28361991514,1cyclictest67750irq/123-lan11:25:053
28361991514,1cyclictest67750irq/123-lan10:05:453
28361991514,1cyclictest67750irq/123-lan07:40:153
28361991414,0cyclictest67750irq/123-lan07:31:553
28361991413,1cyclictest67750irq/123-lan08:00:153
28361991313,0cyclictest67750irq/123-lan09:07:033
28361991111,0cyclictest67750irq/123-lan07:10:153
2835299109,1cyclictest507-21systemd-resolve11:18:211
2835299108,2cyclictest782-21lldpd08:55:461
2835299107,3cyclictest784-21snmpd08:26:081
2835299106,4cyclictest784-21snmpd11:23:351
2835299106,1cyclictest784-21snmpd09:34:371
2834499108,0cyclictest307551latency_hist08:35:000
283569998,1cyclictest27610-21turbostat11:14:552
283529998,1cyclictest784-21snmpd12:04:031
283529998,1cyclictest784-21snmpd11:35:111
283529998,1cyclictest784-21snmpd11:31:321
283529998,1cyclictest784-21snmpd08:40:591
283529997,2cyclictest782-21lldpd09:26:581
283529996,3cyclictest784-21snmpd08:32:021
283529996,3cyclictest782-21lldpd11:25:241
283529996,3cyclictest526-21avahi-daemon10:57:251
283529995,1cyclictest526-21avahi-daemon10:39:581
283619988,0cyclictest67750irq/123-lan07:59:033
283569988,0cyclictest784-21snmpd12:22:222
283569988,0cyclictest784-21snmpd08:21:052
283569988,0cyclictest782-21lldpd10:17:552
283569988,0cyclictest782-21lldpd08:53:122
283569987,1cyclictest784-21snmpd12:31:402
283569987,1cyclictest784-21snmpd10:30:092
283569987,1cyclictest784-21snmpd07:53:162
283569987,1cyclictest784-21snmpd07:27:002
283569987,1cyclictest782-21lldpd08:57:162
283569987,1cyclictest19054-21fschecks_time11:00:112
283569986,2cyclictest784-21snmpd12:37:402
283569986,2cyclictest784-21snmpd11:53:222
283569986,2cyclictest784-21snmpd11:20:222
283569986,2cyclictest784-21snmpd09:24:312
283569986,2cyclictest762-21ntpd07:48:592
283569986,2cyclictest762-21ntpd07:35:312
283569983,3cyclictest29139-21ldconfig10:00:182
283569982,2cyclictest26418-21kworker/u16:3+xprtiod09:39:282
283529988,0cyclictest782-21lldpd09:36:521
283529988,0cyclictest782-21lldpd08:23:471
283529987,1cyclictest784-21snmpd09:02:541
283529986,2cyclictest782-21lldpd12:06:501
283529986,2cyclictest22547-21grep09:20:121
283529985,0cyclictest19489-21cpuspeed_turbos12:10:151
283529983,2cyclictest784-21snmpd11:03:161
283529983,2cyclictest29827-21ssh11:11:001
283529982,3cyclictest245-20systemd-journal10:44:131
283529982,3cyclictest245-20systemd-journal10:44:121
283529982,3cyclictest245-20systemd-journal07:33:511
283449988,0cyclictest784-21snmpd10:23:320
283449988,0cyclictest784-21snmpd07:55:580
283449988,0cyclictest784-21snmpd07:43:550
283449988,0cyclictest782-21lldpd11:18:090
283449988,0cyclictest782-21lldpd10:16:450
283449988,0cyclictest782-21lldpd08:27:510
283449988,0cyclictest191rcu_preempt09:58:470
283449987,1cyclictest784-21snmpd11:50:320
283449987,0cyclictest191rcu_preempt10:32:440
283449986,2cyclictest784-21snmpd09:52:340
283449986,2cyclictest762-21ntpd08:53:340
283449986,2cyclictest762-21ntpd08:08:110
283449986,0cyclictest191rcu_preempt10:04:230
283449985,1cyclictest191rcu_preempt10:11:250
283449984,0cyclictest191rcu_preempt09:40:000
283569977,0cyclictest784-21snmpd10:21:022
283569977,0cyclictest782-21lldpd10:56:072
283569977,0cyclictest782-21lldpd09:18:102
283569977,0cyclictest782-21lldpd07:58:052
283569977,0cyclictest10384-21gcc10:50:182
283569976,1cyclictest784-21snmpd10:26:212
283569976,1cyclictest782-21lldpd10:35:372
283569976,1cyclictest762-21ntpd07:42:272
283569976,1cyclictest17916-21latency_hist09:50:002
283569975,2cyclictest784-21snmpd12:10:132
283569975,2cyclictest784-21snmpd08:26:312
283569975,2cyclictest3115-21cat08:45:142
283569973,3cyclictest15350-21memory12:05:162
283569972,5cyclictest547-21NetworkManager11:43:192
283569970,4cyclictest29172-21df_inode08:30:142
283569970,3cyclictest11227-21sendmail_mailqu12:00:202
283529977,0cyclictest782-21lldpd12:29:331
283529977,0cyclictest782-21lldpd10:33:571
283529976,1cyclictest782-21lldpd07:15:141
283529976,1cyclictest526-21avahi-daemon07:21:511
283529974,0cyclictest526-21avahi-daemon08:46:541
283529973,3cyclictest245-20systemd-journal12:16:281
283449977,0cyclictest784-21snmpd08:03:070
283449976,1cyclictest1246-21needreboot10:05:150
283449976,0cyclictest191rcu_preempt12:07:590
283449976,0cyclictest191rcu_preempt11:46:570
283449976,0cyclictest191rcu_preempt11:00:140
283449975,2cyclictest784-21snmpd09:45:430
283449975,2cyclictest5409-21cat10:45:140
283449975,0cyclictest191rcu_preempt11:41:020
283449975,0cyclictest191rcu_preempt07:49:590
283449975,0cyclictest191rcu_preempt07:45:180
283449973,2cyclictest16-21ksoftirqd/011:55:170
283449971,4cyclictest16-21ksoftirqd/012:03:510
283569966,0cyclictest782-21lldpd08:19:052
283569965,1cyclictest784-21snmpd08:14:522
283569964,2cyclictest1243-21grep10:05:152
283569963,3cyclictest31022-21apt-get08:35:132
283569963,2cyclictest2305-21ssh12:25:282
283569963,2cyclictest20586-21ssh09:51:472
283569963,2cyclictest14444-21ssh09:10:322
283569963,2cyclictest14250-21ssh11:30:132
283569963,2cyclictest12530-21ssh11:28:072
283569963,1cyclictest191rcu_preempt11:05:132
283569961,2cyclictest12497-21dump-pmu-power09:10:002
283569960,5cyclictest7180-21ssh10:11:202
283529966,0cyclictest784-21snmpd11:08:401
283529966,0cyclictest784-21snmpd09:15:031
283529966,0cyclictest784-21snmpd08:38:381
283529965,1cyclictest526-21avahi-daemon09:45:131
283529963,3cyclictest547-21NetworkManager12:24:021
283529962,3cyclictest3803-21sed07:25:261
283529962,3cyclictest29105-21memory07:10:161
283449965,1cyclictest0-21swapper/010:35:180
283449964,2cyclictest28340-21cyclictest08:46:270
283449963,1cyclictest16-21ksoftirqd/010:55:230
283449963,1cyclictest16-21ksoftirqd/008:10:030
283449960,4cyclictest31575-21latency08:35:160
283449960,4cyclictest16-21ksoftirqd/008:20:180
283449960,2cyclictest16-21ksoftirqd/010:50:150
283569955,0cyclictest987-21gltestperf08:40:142
283569955,0cyclictest0-21swapper/209:25:172
283569953,2cyclictest547-21NetworkManager09:55:032
283569953,1cyclictest421ktimers/209:30:192
283569952,2cyclictest19618-21gcc08:05:172
283569951,3cyclictest19593-21open_inodes11:35:192
283569951,1cyclictest0-21swapper/207:10:162
283569950,4cyclictest935-21gltestperf07:20:122
283569950,2cyclictest24293-21gltestperf12:15:142
283569950,2cyclictest17232-21grep08:00:142
283529952,2cyclictest10760-21ssh09:40:421
283569941,2cyclictest9047-21grep09:00:152
283529942,2cyclictest11503-21ld09:05:171
283529941,2cyclictest9481-21ssh10:50:071
283529940,3cyclictest5392-21ntpq08:50:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional