You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-04 - 05:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Mon May 04, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2882199270,0cyclictest0-21swapper/123:08:581
28828992626,0cyclictest0-21swapper/321:46:233
2882899260,2cyclictest8054-21sensors_fan22:21:213
2882899250,24cyclictest22499-21diskmemload23:16:513
2882899250,24cyclictest22499-21diskmemload22:57:093
2882399250,24cyclictest16049-21ssh23:54:062
2882399250,24cyclictest0-21swapper/223:10:112
2882399250,24cyclictest0-21swapper/200:34:272
2882399250,24cyclictest0-21swapper/200:06:192
28821992525,0cyclictest0-21swapper/122:01:591
28816992524,1cyclictest62550irq/127-eno121:53:090
2881699250,25cyclictest0-21swapper/021:45:460
2881699250,24cyclictest22499-21diskmemload23:22:050
2881699250,24cyclictest0-21swapper/022:18:320
2881699250,24cyclictest0-21swapper/022:18:320
28828992421,2cyclictest26532-21diskmemload21:12:403
2882899240,23cyclictest2017-21sh22:45:443
2882899240,23cyclictest0-21swapper/321:57:353
2882899240,0cyclictest0-21swapper/323:59:023
28823992423,1cyclictest0-21swapper/220:21:132
28823992420,4cyclictest0-21swapper/222:21:112
2882399240,24cyclictest0-21swapper/222:36:502
2882399240,23cyclictest8326-21apt-get00:16:142
2882399240,23cyclictest0-21swapper/223:05:572
2882399240,23cyclictest0-21swapper/222:51:132
2882399240,23cyclictest0-21swapper/222:44:192
2882399240,23cyclictest0-21swapper/221:29:262
2882399240,1cyclictest3661-21sh21:50:002
2882399240,1cyclictest0-21swapper/223:16:312
28821992421,2cyclictest22499-21diskmemload00:26:421
2882199240,24cyclictest0-21swapper/123:29:411
2882199240,23cyclictest0-21swapper/121:34:211
2882199240,1cyclictest22499-21diskmemload00:02:501
2882199240,1cyclictest0-21swapper/100:31:171
28816992424,0cyclictest0-21swapper/022:51:450
28816992423,1cyclictest0-21swapper/021:55:160
28816992422,1cyclictest0-21swapper/022:25:330
28816992420,3cyclictest607-21in:imuxsock00:00:010
2881699241,22cyclictest6232-21sh00:13:500
2881699240,23cyclictest24982-21ssh00:30:220
2881699240,23cyclictest0-21swapper/023:25:540
2881699240,0cyclictest24950-21ps21:11:200
2881699240,0cyclictest0-21swapper/023:33:380
28828992323,0cyclictest22499-21diskmemload23:14:223
28828992323,0cyclictest0-21swapper/323:30:013
28828992323,0cyclictest0-21swapper/323:21:503
28828992323,0cyclictest0-21swapper/300:15:193
28828992323,0cyclictest0-21swapper/300:10:433
28828992323,0cyclictest0-21swapper/300:01:503
28828992322,1cyclictest27309-21ssh22:10:543
28828992321,1cyclictest26675-21ssh21:41:173
28828992321,1cyclictest25739-21sh22:36:593
28828992321,1cyclictest20783-21cron20:00:013
28828992321,1cyclictest0-21swapper/321:17:253
2882899230,23cyclictest22499-21diskmemload22:30:583
2882899230,23cyclictest0-21swapper/323:26:593
2882899230,23cyclictest0-21swapper/322:08:223
2882899230,22cyclictest9185-21sed21:26:163
2882899230,22cyclictest34-21ksoftirqd/321:20:083
2882899230,22cyclictest22499-21diskmemload22:25:583
2882899230,22cyclictest22499-21diskmemload21:50:113
2882899230,22cyclictest17205-21rm21:33:063
2882899230,22cyclictest15328-21grep20:55:013
2882899230,22cyclictest14237-21sh00:21:073
2882899230,22cyclictest11215-21sh19:40:003
2882899230,0cyclictest0-21swapper/322:16:593
2882899230,0cyclictest0-21swapper/322:16:593
2882899230,0cyclictest0-21swapper/321:36:273
28823992323,0cyclictest0-21swapper/223:55:112
28823992323,0cyclictest0-21swapper/221:33:442
28823992322,0cyclictest22499-21diskmemload00:11:212
28823992321,1cyclictest15111-21ssh22:56:192
28823992321,1cyclictest0-21swapper/221:24:362
28823992320,2cyclictest22563-21ssh21:37:362
2882399230,23cyclictest0-21swapper/223:21:472
2882399230,23cyclictest0-21swapper/223:04:502
2882399230,22cyclictest23227-21ssh21:10:342
2882399230,22cyclictest22499-21diskmemload23:35:542
2882399230,22cyclictest22499-21diskmemload23:26:042
2882399230,22cyclictest0-21swapper/223:42:432
2882399230,1cyclictest0-21swapper/223:49:132
2882399230,0cyclictest18359-21ssh00:24:232
2882399230,0cyclictest0-21swapper/222:14:542
28821992323,0cyclictest4800-21apt-get21:51:141
28821992323,0cyclictest22499-21diskmemload23:30:121
28821992323,0cyclictest0-21swapper/123:59:341
28821992323,0cyclictest0-21swapper/121:13:361
28821992321,1cyclictest0-21swapper/123:46:171
28821992321,1cyclictest0-21swapper/123:15:071
28821992321,1cyclictest0-21swapper/122:20:591
28821992321,1cyclictest0-21swapper/122:05:451
28821992321,1cyclictest0-21swapper/121:17:181
2882199230,23cyclictest22499-21diskmemload00:08:551
2882199230,23cyclictest0-21swapper/123:40:201
2882199230,23cyclictest0-21swapper/123:12:301
2882199230,23cyclictest0-21swapper/121:36:511
2882199230,22cyclictest7723-21ssh00:15:321
2882199230,22cyclictest32615-21ssh22:15:181
2882199230,22cyclictest32615-21ssh22:15:181
2882199230,22cyclictest26703-21sh23:35:311
2882199230,22cyclictest20835-21munin-run20:00:011
2882199230,22cyclictest0-21swapper/121:25:331
2882199230,0cyclictest0-21swapper/122:48:181
2882199230,0cyclictest0-21swapper/121:49:021
28816992323,0cyclictest0-21swapper/023:17:020
28816992323,0cyclictest0-21swapper/023:10:190
28816992323,0cyclictest0-21swapper/021:30:150
28816992323,0cyclictest0-21swapper/000:00:570
28816992322,1cyclictest0-21swapper/023:40:020
28816992321,1cyclictest6602-21apt-get23:46:140
2881699230,2cyclictest23218-21perf20:05:010
2881699230,2cyclictest19802-21ssh23:01:090
2881699230,2cyclictest13055-21grep21:30:010
2881699230,23cyclictest0-21swapper/022:59:120
2881699230,23cyclictest0-21swapper/022:35:400
2881699230,23cyclictest0-21swapper/021:35:590
2881699230,22cyclictest26780-21sh23:35:370
2881699230,22cyclictest25965-21apt-get20:11:130
2881699230,22cyclictest20797-21sendmail20:00:010
2881699230,22cyclictest1870-21ssh00:38:030
2881699230,22cyclictest18327-21cat22:31:180
2881699230,22cyclictest1340-21latency_hist22:45:000
2881699230,22cyclictest12162-21apt-get23:51:150
2881699230,0cyclictest0-21swapper/023:08:230
2881699230,0cyclictest0-21swapper/022:45:590
2881699230,0cyclictest0-21swapper/022:23:250
28828992221,1cyclictest26566-21ssh23:35:213
28828992220,1cyclictest19457-21ssh00:25:263
2882899220,22cyclictest0-21swapper/323:46:003
2882899220,21cyclictest28492-21ssh00:05:053
2882899220,21cyclictest28290-21sh22:40:073
2882899220,21cyclictest25609-21cron20:10:013
2882899220,21cyclictest25139-21ssh23:05:493
2882899220,21cyclictest10532-21perf20:45:013
2882899220,21cyclictest10372-21ssh22:51:313
2882899220,21cyclictest0-21swapper/319:25:013
2882899220,21cyclictest0-21swapper/300:35:083
2882899220,21cyclictest0-21swapper/300:30:173
2882899220,1cyclictest5332-21cut20:35:003
2882899220,1cyclictest22499-21diskmemload22:00:343
28823992222,0cyclictest0-21swapper/222:30:282
28823992222,0cyclictest0-21swapper/222:15:412
28823992222,0cyclictest0-21swapper/222:15:412
28823992222,0cyclictest0-21swapper/221:40:082
28823992221,1cyclictest16151-21apt-get22:01:142
28823992220,1cyclictest8825-21munin-run19:35:002
2882399221,21cyclictest4053-1kworker/u9:000:36:552
2882399220,21cyclictest9879-21ssh21:55:282
2882399220,21cyclictest31565-21apt-get19:16:132
2882399220,21cyclictest22499-21diskmemload23:30:382
2882399220,21cyclictest0-21swapper/222:45:552
2882399220,21cyclictest0-21swapper/222:05:192
2882399220,1cyclictest25941-21ssh00:02:062
2882399220,1cyclictest19703-21sh00:25:552
2882399220,1cyclictest0-21swapper/222:25:142
2882399220,1cyclictest0-21swapper/221:50:392
28821992221,1cyclictest0-21swapper/123:01:371
28821992220,1cyclictest606-21dbus-daemon19:45:001
28821992220,1cyclictest3270-21apt-get20:31:151
28821992220,1cyclictest13215-21ssh22:55:021
28821992220,1cyclictest0-21swapper/122:30:461
28821992218,2cyclictest313-21systemd-journal20:20:001
2882199220,22cyclictest0-21swapper/123:20:071
2882199220,22cyclictest0-21swapper/122:40:111
2882199220,21cyclictest4366-21apt-get19:26:131
2882199220,21cyclictest28362-21ssh21:42:161
2882199220,21cyclictest0-21swapper/122:36:321
2882199220,21cyclictest0-21swapper/121:20:111
2882199220,21cyclictest0-21swapper/100:20:141
2882199220,1cyclictest2933-21awk20:30:001
2882199220,1cyclictest10824-21df_inode21:56:141
2882199220,1cyclictest0-21swapper/123:50:531
2882199220,1cyclictest0-21swapper/122:26:161
28816992222,0cyclictest22499-21diskmemload00:25:030
28816992220,1cyclictest15788-21ssh22:00:490
2881699220,22cyclictest0-21swapper/022:05:390
2881699220,21cyclictest0-21swapper/022:10:530
2881699220,21cyclictest0-21swapper/000:05:140
2881699220,1cyclictest25685-21ssh21:40:580
2881699220,1cyclictest17377-21ssh00:23:000
2881699220,1cyclictest0-21swapper/021:15:200
2881699220,0cyclictest0-21swapper/000:17:380
2882899210,21cyclictest0-21swapper/323:51:243
2882899210,21cyclictest0-21swapper/323:41:093
2882899210,20cyclictest0-21swapper/320:11:133
2882899210,20cyclictest0-21swapper/319:11:133
2882899210,0cyclictest0-21swapper/323:00:343
2882899210,0cyclictest0-21swapper/320:01:113
2882899210,0cyclictest0-21swapper/319:26:133
28823992121,0cyclictest21173-21apt-get20:01:112
2882399210,20cyclictest30526-21perf20:20:002
2882399210,20cyclictest18084-21apt-get21:01:162
2882399210,20cyclictest0-21swapper/220:36:132
2882399210,1cyclictest0-21swapper/219:56:142
2882199210,21cyclictest0-21swapper/120:11:141
2882199210,21cyclictest0-21swapper/100:10:141
2882199210,0cyclictest0-21swapper/122:50:141
2882199210,0cyclictest0-21swapper/121:01:161
28816992119,1cyclictest0-21swapper/020:40:020
2881699210,20cyclictest29118-21apt-get19:11:130
2881699210,20cyclictest0-21swapper/020:56:120
2881699210,1cyclictest861-21apt-get20:26:140
2881699210,0cyclictest0-21swapper/020:46:140
28828992020,0cyclictest0-21swapper/320:56:123
28828992020,0cyclictest0-21swapper/320:16:143
2882899200,20cyclictest0-21swapper/319:51:143
28823992020,0cyclictest0-21swapper/220:26:132
28823992020,0cyclictest0-21swapper/220:06:142
2882399200,0cyclictest0-21swapper/220:56:122
2882399200,0cyclictest0-21swapper/220:41:142
28821992020,0cyclictest0-21swapper/121:06:141
28821992020,0cyclictest0-21swapper/120:21:131
28821992020,0cyclictest0-21swapper/119:51:131
28821992020,0cyclictest0-21swapper/119:20:021
28821992018,1cyclictest15661-21apt-get20:56:121
2882199200,20cyclictest0-21swapper/120:40:011
2882199200,0cyclictest0-21swapper/119:36:141
28816992020,0cyclictest0-21swapper/020:31:140
28816992020,0cyclictest0-21swapper/020:16:140
28816992020,0cyclictest0-21swapper/019:21:130
28816992019,0cyclictest0-21swapper/019:26:130
2881699200,19cyclictest622-21gdbus19:40:000
2881699200,0cyclictest20476-21apt-get21:06:140
2881699200,0cyclictest0-21swapper/019:51:130
28828991919,0cyclictest13969-21apt-get19:46:133
28828991919,0cyclictest0-21swapper/321:06:133
28828991917,1cyclictest11575-21apt-get19:41:133
28816991919,0cyclictest0-21swapper/021:01:160
28816991917,1cyclictest16015-21turbostat.cron19:50:000
28816991916,2cyclictest7732-21idleruntime-cro20:40:010
2882899182,11cyclictest34-21ksoftirqd/320:26:133
28828991818,0cyclictest0-21swapper/319:31:143
28823991818,0cyclictest0-21swapper/220:11:142
2882399180,1cyclictest22495-21cron21:10:002
28821991817,1cyclictest23571-21apt-get20:06:141
28821991817,1cyclictest13267-21apt-get20:51:131
2882899170,17cyclictest0-21swapper/320:36:133
28823991717,0cyclictest0-21swapper/219:26:132
28823991717,0cyclictest0-21swapper/219:21:132
28823991715,1cyclictest15305-21turbostat20:55:012
2882399170,1cyclictest16005-21cron19:50:002
28821991717,0cyclictest0-21swapper/120:46:141
28821991717,0cyclictest0-21swapper/120:41:141
28821991717,0cyclictest0-21swapper/119:16:131
28816991715,1cyclictest31565-21apt-get19:16:130
2882399160,2cyclictest9600-21/usr/sbin/munin19:36:162
28821991614,1cyclictest0-21swapper/119:50:001
28816991614,1cyclictest25630-21munin-run20:10:010
28816991614,1cyclictest0-21swapper/019:45:000
28821991515,0cyclictest0-21swapper/120:01:111
28821991515,0cyclictest0-21swapper/119:11:131
28828991414,0cyclictest0-21swapper/321:00:093
28828991414,0cyclictest0-21swapper/320:45:253
2882899140,14cyclictest0-21swapper/319:15:273
2882899140,0cyclictest0-21swapper/320:20:483
2882399140,14cyclictest0-21swapper/220:45:122
2882399140,14cyclictest0-21swapper/220:30:462
2882399140,14cyclictest0-21swapper/219:10:582
2882399140,0cyclictest0-21swapper/219:50:052
2882399140,0cyclictest0-21swapper/219:40:322
28821991414,0cyclictest0-21swapper/119:30:161
2881699140,14cyclictest0-21swapper/020:50:160
2881699140,14cyclictest0-21swapper/020:20:380
2881699140,14cyclictest0-21swapper/019:30:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional